Multi display apparatus and multi display method

ABSTRACT

A multi display apparatus includes a first body including a first display, a second body including a second display, a hinge configured to rotatably connect the first body and the second body to support the first body and the second body, a sensor configured to sense a first user manipulation made in a nondecisive location of a nonstructured first screen displayed on the first display and to sense a second user manipulation made in a nondecisive location of a nonstructured second screen displayed on the second display, and a controller configured to combine the sensed first user manipulation and second user manipulation to perform an operation corresponding to a result of the combination.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application filed under 37 C.F.R.1.53(b) of U.S. patent application Ser. No. 14/050,936, filed Oct. 10,2013, pending, which claims priority under 35 U.S.C. §119(a) from KoreanPatent Application No. 10-2012-0112638, filed Oct. 10, 2012, thedisclosures of all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

Methods and apparatuses consistent with the exemplary embodiments of thepresent general inventive concept relate to a multi display apparatus,and more particularly to a multi display apparatus configured to receiveinputs through each display thereof and to perform outputtingaccordingly, and a multi display method thereof.

2. Description of the Related Art

A portable electronic apparatus in recent use such as a smart phonegenerally has one display.

However, as the performance of a Central Processing Unit (CPU) mountedin a portable electronic apparatus increases and multi-taskingenvironments capable of executing various applications at the same timeare provided, it becomes difficult to effectively utilize theperformance a display apparatus having just one display.

In addition, due to the development of battery technologies and thermaltreatment systems, the design paradigm that a portable electronicapparatus may only have one display is not an acceptable truth any more.

Furthermore, the need has been raised to expand touch screens andintroduce more various user experiences to portable electronicapparatuses in user experience oriented interface environment.

Against this background, portable electronic apparatuses having two ormore displays, that is, multi display apparatuses, are emerging.

A multi display apparatus is receiving attention as a next generationdisplay apparatus since it may execute and display various applicationson multiple displays based on powerful hardware performance, and thus isnot only appropriate to a multi-tasking environment, but also providesvarious and abundant information to a user, thereby providing new userexperience.

In a multi display apparatus, not only outputting but also inputting maybe performed through a plurality of displays. Therefore, the need hasbeen raised for the technology which may perform more various inputs andoutputs according to inputs of each display in a multi displayapparatus.

SUMMARY OF THE INVENTION

Exemplary embodiments of the present general inventive concept provide amulti display apparatus configured to perform an unstructured inputthrough each display of the multi display apparatus, and to combine eachinput to perform an output corresponding thereto, and a multi displaymethod thereof.

Additional features and utilities of the present general inventiveconcept will be set forth in part in the description which follows and,in part, will be obvious from the description, or may be learned bypractice of the general inventive concept.

Exemplary embodiments of the present general inventive concept provide amulti display apparatus including a first body including a firstdisplay, a second body including a second display, a hinge configured torotatably connect the first body and the second body to support thefirst body and the second body, a sensor configured to sense a firstuser manipulation made in a nondecisive location of a nonstructuredfirst screen displayed on the first display and to sense a second usermanipulation made in a nondecisive location of a nonstructured secondscreen displayed on the second display, and a controller configured tocombine the sensed first user manipulation and the sensed second usermanipulation to perform an operation corresponding to a result of thecombination.

The controller may change a function matching the second usermanipulation according to the first user manipulation, and thecontroller may perform an operation corresponding to the second usermanipulation according to the changed function, when the second usermanipulation is sensed.

The second user manipulation may be a line input by a drag operation,the controller may increase a thickness or transparency of the lineinput of the second user manipulation when the first user manipulationis a pinch out input, the controller may decrease a thickness ortransparency of the line input of the second user manipulation when thefirst user manipulation is a pinch in input, and the controller maychange a texture or thickness of the line input of the second usermanipulation when the first user manipulation is a touch and rotateinput.

The first user manipulation may be a touch and hold input and the seconduser manipulation may be a touch input, and the controller may display amenu corresponding to the second user manipulation when the first usermanipulation and the second user manipulation are sensed at the sametime.

The controller may change the second screen displayed on the seconddisplay according to the first user manipulation when the first usermanipulation is sensed, and the controller may display the second screenchanged by the first user manipulation according to the second usermanipulation, when the second user manipulation is sensed.

The controller may expand the second screen displayed on the seconddisplay when the first user manipulation is a pinch out input, thecontroller may reduce the second screen displayed on the second displaywhen the first user manipulation is a pinch in input, and the controllermay rotate the second screen displayed on the second display when thefirst user manipulation is a touch and rotate input.

Exemplary embodiments of the present general inventive concept alsoprovide a multi display apparatus including a first body including afirst display, a second body including a second display, a hingeconfigured to rotatably connect the first body and the second body tosupport the first body and the second body, a sensor configured to sensea first user manipulation made in a nondecisive location of the firstdisplay and a second user manipulation made in a nondecisive location ofthe second display, when the first display and the second displayrespectively display a nonstructured first screen and a nonstructuredsecond screen, and a controller configured to combine the sensed firstuser manipulation and second user manipulation to recognize thecombination as one user gesture, and to display an execution screencorresponding to the one user gesture on at least one of the firstdisplay and the second display.

The first user manipulation may be an input proximate to or touching thefirst display using a user's body, and the second user manipulation maybe an input proximate to or touching the second display using an inputpen.

Exemplary embodiments of the present general inventive concept alsoprovide a multi display method of a multi display apparatus including afirst body including a first display, a second body including a seconddisplay, and a hinge configured to rotatably connect the first body andthe second body to support the first body and the second body, the multidisplay method including displaying a nonstructured first screen on thefirst display, displaying a nonstructured second screen on the seconddisplay, sensing a first user manipulation made in a nondecisivelocation on the first screen and a second user manipulation made in anondecisive location on the second screen, and combining the sensedfirst user manipulation and the sensed second user manipulation andperforming an operation corresponding to a result of the combination.

Performing the operation may include changing a function matching thesecond user manipulation according to the first user manipulation, andperforming an operation corresponding to the second user manipulationaccording to the changed function when the second user manipulation issensed.

The second user manipulation may be a line input by a drag operation.Changing the function may include increasing a thickness of the lineinput of the second user manipulation when the first user manipulationis a pinch out input, reducing a thickness of the line input of thesecond user manipulation when the first user manipulation is a pinch ininput, and changing a texture of the line input of the second usermanipulation when the first user manipulation is a touch and rotateinput.

The first user manipulation may be a touch and hold input and the seconduser manipulation may be a touch input. Performing the operation mayinclude displaying a menu corresponding to the second user manipulationwhen the first user manipulation and the second user manipulation aresensed at the same time.

The multi display method may further include changing the second screendisplayed on the second display according to the first usermanipulation, when the first user manipulation is sensed. Performing theoperation may include displaying the second screen changed by the firstuser manipulation according to the second user manipulation, when thesecond user manipulation is sensed.

The changing the second screen may include expanding the second screendisplayed on the second display when the first user manipulation is apinch out input, reducing the second screen displayed on the seconddisplay when the first user manipulation is a pinch in input, androtating the second screen displayed on the second display when thefirst user manipulation is a touch and rotate input.

Exemplary embodiments of the present general inventive concept alsoprovide a multi display method of a multi display apparatus including afirst body including a first display, a second body including a seconddisplay, and a hinge configured to rotatably connect the first body andthe second body to support the first body and the second body, the multidisplay method including sensing a first user manipulation made in anondecisive location of a nonstructured first screen displayed on thefirst display, sensing a second user manipulation made in a nondecisivelocation of a nonstructured second screen displayed on the seconddisplay, and combining the sensed first user manipulation and seconduser manipulation to recognize the combination as one user gesture, anddisplaying an execution screen corresponding to the one user gesture onat least one of the first display and the second display.

The first user manipulation may be an input proximate to or touching thefirst display using a user's body, and the second user manipulation maybe an input proximate to or touching the second display using an inputpen.

The controller may extract contents displayed on the first display whenthe first user manipulation is sensed, and display a screencorresponding to the extracted contents on the second display when thesecond user manipulation is sensed.

The controller may capture a screen displayed on the first displayaccording to the first user manipulation, and display the capturedscreen on a location on the second display where the second usermanipulation is made.

The controller may extract contents displayed on the first display andthe second display and display a screen corresponding to the extractedcontents on the second display a third user manipulation is sensed on anondecisive location of the second screen while the first usermanipulation is sensed.

Performing the operation may include extracting contents displayed onthe first display when the first user manipulation is sensed, anddisplaying a screen corresponding to the extracted contents on thesecond display when the second user manipulation is sensed.

Extracting contents may include capturing a screen displayed on thefirst display according to the first user manipulation, and displayingthe screen corresponding to the extracted contents may includedisplaying the captured screen on the second display on a location wherethe second user manipulation is made.

A non-transitory computer-readable recording medium may containcomputer-readable codes as a program to execute the multi displaymethod.

Exemplary embodiments of the present general inventive concept alsoprovide a multi display apparatus including a plurality of displays, asensor configured to sense a first user manipulation on a first displayof the plurality of displays and to sense a second user manipulationmade in a nondecisive location on at least one second display of theplurality of displays, and a controller to perform an operation on atleast one of the plurality of displays according to a combination of thesensed first user manipulation and the sensed second user manipulation.

The sensor may also be configured to sense a location of each of theplurality of displays relative to each other.

The second user manipulation may be made in a nondecisive location on aplurality of second displays, and the controller may perform theoperation on each of the plurality of second displays on which thesecond user manipulation is made.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other features and utilities of the present generalinventive concept will become apparent and more readily appreciated fromthe following description of the embodiments, taken in conjunction withthe accompanying drawings of which:

FIG. 1 is a block diagram illustrating a configuration of a multidisplay apparatus according to an exemplary embodiment of the presentgeneral inventive concept;

FIG. 2 is a flowchart illustrating operations of a multi displayapparatus according to an exemplary embodiment of the present generalinventive concept;

FIG. 3 is a view illustrating a structured screen according to anexemplary embodiment of the present general inventive concept;

FIG. 4 is a flowchart illustrating an operation of a multi displayapparatus according to an exemplary embodiment of the present generalinventive concept configured to change a function matching a second usermanipulation, when there is a first user manipulation;

FIG. 5 is a reference view illustrating a display screen of an exemplaryembodiment of the multi display apparatus of FIG. 4;

FIG. 6 is a flowchart illustrating another exemplary embodiment of themulti display apparatus of FIG. 4;

FIG. 7 is a reference view illustrating a display screen of an exemplaryembodiment of the multi display apparatus of FIG. 6;

FIG. 8 is a flowchart illustrating another exemplary embodiment of themulti display apparatus of FIG. 4;

FIG. 9 is a reference view illustrating a display screen of an exemplaryembodiment of the multi display apparatus of FIG. 8;

FIG. 10 is a flowchart illustrating another exemplary embodiment of themulti display apparatus of FIG. 2;

FIG. 11 is a reference view illustrating a display screen of anexemplary embodiment of the multi display apparatus of FIG. 10;

FIG. 12 is a flowchart illustrating operations of a multi displayapparatus according to an exemplary embodiment of the present generalinventive concept;

FIG. 13 is a flowchart illustrating an exemplary embodiment of the multidisplay apparatus of FIG. 12;

FIG. 14 is a reference view illustrating a display screen of anexemplary embodiment of the multi display apparatus of FIG. 13;

FIG. 15 is a flowchart illustrating another exemplary embodiment of themulti display apparatus of FIG. 12;

FIG. 16 is a reference view illustrating a display screen of anexemplary embodiment of the multi display apparatus of FIG. 15;

FIG. 17 is a flowchart illustrating another exemplary embodiment of themulti display apparatus of FIG. 12;

FIG. 18 is a reference view illustrating a display screen of anexemplary embodiment of the multi display apparatus of FIG. 17;

FIG. 19 is a flowchart illustrating an operation of a multi displayapparatus according to an exemplary embodiment of the present generalinventive concept;

FIG. 20 is a flowchart illustrating an exemplary embodiment of the multidisplay apparatus of FIG. 19;

FIGS. 21 to 23 are views illustrating a display screen of an exemplaryembodiment of the multi display apparatus of FIG. 20;

FIG. 24 is a flowchart illustrating a specific another exemplaryembodiment of the multi display apparatus of FIG. 19;

FIG. 25 is a reference view illustrating a display screen of anexemplary embodiment of the multi display apparatus of FIG. 24;

FIG. 26 is a block diagram illustrating a configuration of a multidisplay apparatus according to an exemplary embodiment of the presentgeneral inventive concept;

FIG. 27 is a block diagram illustrating a hardware configuration of acontroller according to an exemplary embodiment of the present generalinventive concept;

FIG. 28 is a view illustrating a system hierarchy structure of a multidisplay apparatus according to an exemplary embodiment of the presentgeneral inventive concept;

FIG. 29 is a view illustrating a circuit configuration of an imageoutputter of the first display or the second display according to anexemplary embodiment of the present general inventive concept;

FIG. 30 is a view illustrating a circuit structure configuring R, G, andB pixel areas within the display panel of FIG. 29;

FIG. 31 is a cross-sectional view of a first display or second displayincluding a touch screen according to an exemplary embodiment of thepresent general inventive concept;

FIG. 32 is a block diagram illustrating a display driver of a multidisplay apparatus according to an exemplary embodiment of the presentgeneral inventive concept;

FIG. 33 is a block diagram illustrating a configuration of a displaydriver according to another exemplary embodiment of the present generalinventive concept;

FIG. 34 is a view illustrating a proximate touch according to anexemplary embodiment of the present general inventive concept;

FIG. 35 is a detailed perspective view of a multi display apparatusaccording to an exemplary embodiment of the present general inventiveconcept;

FIGS. 36 to 38 are perspective views illustrating states of a multidisplay apparatus changed in various ways with reference to a hingeaccording to an exemplary embodiment of the present general inventiveconcept;

FIGS. 39 and 40 are perspective views illustrating standing states of amulti display apparatus according to an exemplary embodiment of thepresent general inventive concept;

FIGS. 41 and 42 are views illustrating an arrangement of two cameras ofa multi display apparatus according to an exemplary embodiment of thepresent general inventive concept;

FIGS. 43 and 47 are flowcharts illustrating a multi display methodaccording to various exemplary embodiments of the present generalinventive concept;

FIG. 48 is a view illustrating an arrangement of displays of the multidisplay apparatus wirelessly connected according to another exemplaryembodiment of the present general inventive concept; and

FIGS. 49-51 are tables illustrating exemplary gestures.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the embodiments of the presentgeneral inventive concept, examples of which are illustrated in theaccompanying drawings, wherein like reference numerals refer to the likeelements throughout. The embodiments are described below in order toexplain the present general inventive concept while referring to thefigures.

In the present specification, a multi display apparatus is an apparatushaving a plurality of displays including one or more touch screens, andconfigured to execute an application or display contents. Examplesinclude a tablet personal computer (PC), portable multimedia player(PMP), personal digital assistant (PDA), smart phone, mobile phone, anddigital photo frame, etc. The exemplary embodiments of the presentgeneral inventive concept described below and illustrated in thedrawings are directed to a multi display apparatus such as a cellularphone or smart phone. However, it will be understood that the presentgeneral inventive concept is not limited to these exemplary embodiments,and may be applied to any multi display apparatus.

FIG. 1 is a block diagram illustrating a configuration of a multidisplay apparatus 100 according to an exemplary embodiment of thepresent general inventive concept, and FIG. 2 is a flowchartillustrating an operation of a multi display apparatus 100 according toan exemplary embodiment of the present general inventive concept.

With reference to FIG. 1, a multi display apparatus 100 according to anexemplary embodiment of the present general inventive concept includes afirst body 2, a second body 4, a hinge 185, a sensor 150, and acontroller 130.

The first body 2 may include a first display 190 a of the multi displayapparatus 100. In addition, as illustrated in FIG. 1, the first body 2may include the sensor 150, and the controller 130 to be explainedhereinbelow. However, the sensor 150 and controller 130 mayalternatively be included in the second body 3. Besides the above, thefirst body 2 may further include circuit components (not illustrated) todrive the first display 190 a and a bezel (not illustrated) to acceptthe circuit components, but further configuration and explanation isomitted for convenience of explanation.

The second body 4 may include a second display 190 b of the multidisplay apparatus 100. Besides the above, the second body 4 includescircuit components (not illustrated) to drive the second display 190 band a bezel (not illustrated) to accept the circuit components. Thecircuit components may be included in either one of the first body 2 orthe second body 4, or in both the first body 2 and the second body 4.The first body 2 and the second body 4 are connected to each other bythe hinge 185, to be explained hereinbelow.

The first display 190 a and the second display 190 b may be referred toas a multi display (dual display) 190.

The hinge 185 is configured to connect the first body 2 and the secondbody 4 to support the first body 2 and the second body 4 so that theyare rotatable. More specifically, the hinge 185 combines the first body2 and the second body 4 physically so that they could be connected toeach other, and plays the role of a shaft so that the first body 2 andthe second body 4 may rotate while they are connected to each other. Anexterior configuration of the first and second body 4 and the hinge 185will be explained hereinbelow with reference to the figures. However,the hinge 185 may be embodied as a part of a flexible connector orflexible touch screen (not illustrated) besides the structure to beexplained hereinbelow.

The sensor 150 is configured to sense a user manipulation of the multidisplay apparatus 100. For example, when there is a touch input of auser object 50 (illustrated for example in FIG. 34) through a display190 a, 190 b, the sensor 150 senses a location (coordinate) of a touchinput and a type of input. Especially, when the first display 190 a andthe second display 190 b display a nonstructured first screen and secondscreen, respectively, as described below with reference to FIG. 2, thesensor 150 senses a first user manipulation regarding a nondecisivelocation of the first screen and a second user manipulation regarding anondecisive location of a second screen.

A structured state refers to a state where various menus and otherobjects that a user may select are arranged according to a predeterminedlayout. That is, a structured screen is a screen configured to providean interface structured by such a layout. Examples of a structuredscreen include templates 390 having various shapes, keypads 391 wherenumbers or text data could be input, and screens where tool bars 392,menu buttons 393, scroll bars 394, etc. are displayed, as illustrated inFIG. 3. A user becomes able to perform an input by simply touching abutton of a menu at a predetermined location on a screen or by anoperation (for example, dragging their finger) while touching thebutton. On the other hand, a nonstructured screen refers to a screenthat does not provide a structured interface. When a nonstructuredscreen is displayed, the user may simply drag a nondecisive location ofthe screen or may touch for a predetermined time (touch & hold input)and perform an input. Various types of user input will be explainedhereinbelow in tables 1 to 3.

Herein, a nondecisive location denotes any location on the screen. Inother words, a nondecisive location means that a same result can beobtained regardless of the location of an input on the screen.

In various exemplary embodiments of the present general inventiveconcept, the sensor 150 may sense whether or not a user object 50touched or is proximate to the multi display apparatus 100, and thelocation where the touch or the proximity is made. The definitions of“touch” and “proximate” for the purposes of the present generalinventive concept are given below. In addition, the sensor 150 maydistinguish a case where the user object 50 is the user's body and acase where the user object 50 is an input pen 200. Technological meanswhich may support the aforementioned exemplary embodiments will beexplained in further detail hereinbelow.

The controller 130 combines a first user manipulation and a second usermanipulation, and performs a control operation corresponding to a resultof that combination. For example, when there is a touch input in thefirst display 190 a and there is also a proximate input in the seconddisplay 190 b, the controller 130 may perform operations correspondingto each of the touch input and the proximate input, but the controller130 may also perform a calculation corresponding to the result of thatcombination. In addition, the controller 130 may display informationcorresponding to at least one calculation result of the first display190 a and the second display 190 b. Operations of the controller 130will be explained in further detail hereinbelow based on each exemplaryembodiment of the present general inventive concept, and thetechnological means configuring the controller 130 may be explained infurther detail hereinbelow.

First of all, various input types to be used in the exemplaryembodiments of the present general inventive concept are as follows.Herein, inputs using the user object 50 include a touch, proximity, andmotion. The user object 50 may be part of the user's body such asfingers or palm, or other objects such as an input pen 200. The inputpen 200 may be made from conductive material such as metal, but it mayalso be made from other materials that may be detected by the sensor150. For example, an input pen 200 may be configured to include magneticcoil, and the sensor 150 of the multi display apparatus 100 may includea magnetic sensor.

As such, the input pen 200 including the magnetic coil may also be theuser object 50. As such, the user object 50 includes all types ofobjects and bodies that may be recognized within contact or apredetermined range. These various objects may be used independently orused in combinations.

A touch denotes an input of contacting the aforementioned user object50. For example, it is a user's operation of contacting one location ora plurality of consecutive locations on a touch screen 192 (illustratedin FIG. 31) using a user object 50 may include the user's fingers(especially, index finger) of a left hand and right hand, thumb, or astylus pen 200 contacting the touch screen 192. Technological means torecognize a touch will be explained hereinbelow.

Proximity denotes an input of locating a user object 50 within thepredetermined range 5 of the sensor 150 of the multi display apparatus100 without directly touching or pressing the multi display apparatus100. Various exemplary embodiments of the present general inventiveconcept presuppose differentiating between the aforementioned proximityand touch. Technological means to recognize proximity will be explainedin further detail hereinbelow.

A motion denotes an input of a predetermined identifiable movement whileproximate to the multi display apparatus 100. “Motion” may be includedin the definition of “proximity,” and thus a detailed explanation isomitted herein.

Various input types used in the present disclosure are explained withreference to the table below. The explanation below is made based on atouch input, but proximate inputs may be based on the same reference. Inaddition, as aforementioned, the user object 50 may be the user's bodypart or input pen 200.

Referring to FIG. 49, which includes Table 1, Table 1 illustratesexemplary gesture types which use one finger. Although hereinbelow isexplanation in the case where the user object 50 is a finger, caseswhere other user objects 50 are used are also defined as the sameoperation.

With reference to table 1, examples of a gesture type using one fingerinclude Tap, Touch & Hold, Double Tap, Drag, Drag & Drop, and Flick etc.

A tap denotes an operation where the user contacts the user object 50 tothe screen for a predetermined time and then separates it therefrom. Atouch & hold is an operation where the user object 50 touches the touchscreen 192 for more than a predetermined time. A double tap denotes anoperation of performing a tap twice quickly and consecutively within apredetermined time. A drag is an operation of moving to a predetermineddirection while a touch is made, and a drag & drop is an operation oftouching any object on the touch screen 192 and performing a drag to apredetermined location and then separating the user object 50 therefrom.A flick is an operation of quickly dragging.

Referring to FIG. 50, which includes Table 2, Table 2 illustratesexemplary gesture types using two fingers. Although hereinbelow isexplanation in the case where the user object 50 is a finger, caseswhere other user objects 50 are used are also defined as the sameoperation.

With reference to table 2, gesture types using two fingers include a TwoFinger Tap, Touch & Spread, Pinch Out, Pinch In, Two Finger Drag, CrossTwo Finger, and Touch & Rotate etc.

A Two Finger Tap is an operation of two fingers tapping at the sametime. A Touch & Spread is an operation of two fingers pressing the touchscreen 192 at the same time, one finger moving, and the other fingermoving in a straight line. A Pinch Out is an operation of two fingerstouching the screen at the same time and then dragging away from eachother, while a Pinch In is an operation of two fingers touching thetouch screen 192 and then dragging in a direction towards each other. ATwo Finger Drag is an operation of two fingers dragging in a samedirection, a Cross Two Finger is an operation of dragging in anapproaching direction at the same time and then dragging away from eachother again. Finally, a Touch & Rotate is an operation with one fingertouching the touch screen 192 (touch & hold) and the other rotatingaround the still finger.

Referring to FIG. 51, which includes Table 3, Table 3 illustratesexemplary gesture types using two or more fingers and a gesture typeusing a palm.

With reference to table 3, examples of a gesture type using two or morefingers include a Three Finger Touch, Four Finger Touch, and Five FingerTouch etc. In addition, it is possible to perform gesture operationssuch as a Tap, Drag, and Rotate, as illustrated in the aforementionedtables 1 and 2.

The sensor 150 identifies each input type of the aforementioned tables 1to 3, and transmits the identification information to the controller130. Each aforementioned input type may be sensed independently and betransmitted to the controller 130. The controller 130 may processcalculations corresponding to combination results of each input type ora plurality of input types sensed by the sensor and display informationcorresponding to the calculation results on the first display 190 a orsecond display 190 b.

Operations of the aforementioned multi display apparatus 100 may beperformed according to the flowchart illustrated in FIG. 2.

First of all, the first display 190 a and second display 190 b eachdisplays the nonstructured first screen and second screen (operationS210). In this state, the multi display apparatus 100 may sense a firstuser manipulation made in a nondecisive location of the first screen anda second user manipulation made in a nondecisive location of a secondscreen (operations S220, S230). In FIG. 2, it is explained that thefirst user manipulation is made first and then the second usermanipulation is made next, but the first and second user manipulationsmay be made independently and sensed. When the first and second usermanipulations are each sensed, the multi display apparatus 100 combinesthe sensed first user manipulation and second user manipulation(operation S240), and performs one or more operations corresponding tothat combination result (operation S250). Operations performed by thecombination of the user manipulations may be embodied in various waysaccording to exemplary embodiments of the present general inventiveconcept.

Hereinbelow is explanation on various examples of operations performedby the first and second user manipulations made in nondecisivelocations.

First of all, hereinbelow is explanation of an exemplary embodiment ofchanging a function matching a second user manipulation, when there is afirst user manipulation through the first display 190 a of the multidisplay apparatus 100.

In a case of performing a user manipulation on a display to perform aparticular operation, it is necessary to set a function matching theuser manipulation. For example, in a case of performing an operation ofdrawing a line according to a user manipulation on a screen of adisplay, there may be a need to adjust the thickness of the lineaccording to circumstances. Herein, it is possible to use a menu screento adjust the thickness of the line through a structured interface andadjust the thickness of the line. However, such an operation iscumbersome, and may cause inconvenience if there is frequent changing offunctions. Therefore, there is a need to reduce the setting process foruser manipulation.

FIG. 4 is a flowchart illustrating operations of the multi displayapparatus 100 changing functions matching the second using manipulationwhen there is a first user manipulation, and FIG. 5 is a reference viewillustrating the display screen of the multi display apparatus 100 ofFIG. 4.

With reference to FIG. 4, first of all, the sensor 150 senses a pointtouched in the first display 190 a or the second display 190 b andnotifies the controller 130. The controller 130 determines whether apinch in or pinch out is made according to the number and location ofthe touch point (operation S410). The controller 130 determines a pinchout when a touch is sensed at two or more points and the straight linedistance between the sensed coordinates increases, and determines apinch out when the straight line distance between the sensed coordinatesdecreases. The pinch in and pinch out may be the first user manipulationaforementioned.

The controller 130 may perform operations corresponding to thedetermined pinch input. That is, as illustrated in FIG. 4, it ispossible to adjust the thickness of the line or change the transparencyor other characteristics according to the pinch input.

In this state, when a drag is made in a nondecisive location of anotherdisplay, through the sensor 150 the controller 130 may sense a touchinput by a drag defining a drag route (operation S420). Herein, the dragmay be the aforementioned second user manipulation.

The controller 130 may recognize, according to the detected pinch in orpinch out of the first user input, whether a command to adjustcharacteristics of the line, such as thickness or transparency, has beeninput (operation S430). The controller 130 performs an operation ofdisplaying a line along the drag route according to the convertedcharacteristics (operation S440).

View (1) of FIG. 5 is a view illustrating a state of adjusting thethickness of a line.

According to view (1) of FIG. 5, the first display 190 a, located on theleft, displays the nonstructured first screen, and the sensor 150 sensesthe first user manipulation at the nondecisive location of the firstscreen. Such an operation is performed by a touch sensor, but this willbe explained in further detail below. In addition, the second display190 b, located on the right, also displays the nonstructured secondscreen. It will be understood that the first and second displays 190 aand 190 b are not necessarily located on the left and right asillustrated in FIG. 5, and may have different orientations to each otherdepending on the configuration of the multi display apparatus 100.

As illustrated in FIG. 5, the user may perform a touch input such as apinch in or pinch out using fingers in the first display 190 a. At thesame time, the user may perform a touch input using an input pen 200 orone's fingers, or other user objects 50 on the second display 190 b. Forconvenience of explanation, FIG. 5 illustrates a case of an input pen200, and also in various exemplary embodiments of the present generalinventive concept to be explained below, explanation is based on alimitation that manipulations are made using an input pen 200 regardingone display, and using the user's body regarding the other display.However, as aforementioned, manipulations may obviously be made usingonly the input pen 200 or only the user's body regarding both displays.Explanation hereinbelow will be made in consideration of specificsituations and convenience of explanation, but the exemplary embodimentsof the present general inventive concept may be performed in all typesof manipulation methods.

In the exemplary embodiment of view (1) of FIG. 5, when there is a touchinput by a pinch in using fingers on the first display 190 a, thecontroller 130 displays a line according to the changed thickness alongthe input route of the input pen 200 (or user's body) regarding thesecond display 190 b. Therefore, a gradually thinner line is displayedalong the drag route of the input pen 200 on the second display 190 b.On the other hand, when there is a touch input by a pinch out usingfingers on the first display 190 a, the controller 130 displays a lineaccording to the changed thickness along the input route of the inputpen 200 regarding the second display 190 b. Accordingly, a graduallythicker line is displayed along the drag route of the input pen 200 onthe second display 190 b.

View (2) of FIG. 5 illustrates an exemplary embodiment of the presentgeneral inventive concept similar to (view 1), illustrating operationsof the multi display apparatus 100 of changing the transparency of theline output by the second user manipulation when the first usermanipulation is a touch input by a pinch in or pinch out. For example,in a case where the first user manipulation is a touch input by a pinchin, when performing an operation of drawing a line on the second display190 b with the input pen 200, the transparency of the line being drawnby the input pen 200 decreases. On the other hand, when the first usermanipulation is a touch input by a pinch out, the transparency of theline being drawn by the input pen 200 increases. Of course, the outputscorresponding to the pinch in and pinch out may be set to be opposite toeach other from this particular exemplary embodiment of the presentgeneral inventive concept.

In this exemplary embodiment of the present general inventive concept,the user becomes able to change the thickness or transparency of aninput expression regarding the second display 190 b with simply a touchinput by a pinch in or pinch out into the first display 190 a of themulti display apparatus 100 only. That is, there is no need to performnumerous manipulations using an additional structured interface in orderto change the functions matching the second user manipulation. Byperforming an input at a certain point on a nonstructured screen, theuser becomes able to easily change the function corresponding to theuser manipulation.

FIG. 6 is a flowchart illustrating another exemplary embodiment of themulti display apparatus 100 of FIG. 4, and FIG. 7 is a reference viewillustrating a display screen of an exemplary embodiment of the multidisplay apparatus 100 of FIG. 6.

With reference to FIG. 6, the sensor 150 senses a first usermanipulation which is maintained for a predetermined time with aninitial touch made in the first display 190 a and the touch is notremoved, that is, a touch input by a touch & hold (operation S610). Thatis, the display 190 a displays the nonstructured first screen, and thesensor 150 senses that there is a touch input by a touch & hold at anondecisive location of the first screen. Such an operation may beperformed by a touch sensor, but this will be explained in more detailbelow.

After the first user manipulation is sensed, a second user manipulationmay be performed on the second display 190 b while the first usermanipulation is maintained. This second user manipulation may be a touchinput by a drag on the second display 190 b (operation S620). This dragoperation defines a drag route on the second display 190 b.

In this case, the controller 130 recognizes that the first usermanipulation is to convert the tools corresponding to the second usermanipulation, and that the second user manipulation is an operation toperform an input along the drag route (operation S630). In addition, thecontroller 130 performs an input along the drag route according to thechanged tool (operation S640). In the exemplary embodiment of thepresent general inventive concept illustrated in FIG. 7, when there is atouch input by a touch & hold using fingers in the first display 190 a,the controller 130 converts the tool corresponding to the input of theinput pen 200 regarding the second display 190 b into an eraser. Inaddition, the controller 130 recognizes that there is an eraser inputalong the drag route of the input pen 200 on the second display 190 b,and deletes the display on the route.

Unlike the above, when the user stops a touch & hold input regarding thefirst display 190 a, that is, when the user detaches one's hand from thefirst display 190 a, the sensor 150 does not sense the first usermanipulation any more, but senses only the second user manipulation, andthus the controller 130 performs only operations corresponding to thesecond user manipulation. As an example, in the exemplary embodiment ofthe present general inventive concept illustrated in FIG. 4, the toolcorresponding to the input of the input pen 200 returns from an eraserto the original tool again, and an input is performed according to theoriginal tool.

In such an exemplary embodiment of the present general inventiveconcept, the user becomes able to change the input tool regarding thesecond display 190 b with only a touch input (such as for example atouch & hold) in the first display 190 a. That is, it is not necessaryto perform numerous manipulations using an additional structuredinterface in order to change the function matching the second usermanipulation. The user becomes able to easily change the functioncorresponding to the user manipulation by performing an input at anypoint on the nonstructured screen.

FIG. 8 is a flowchart illustrating another exemplary embodiment of themulti display apparatus 100 of FIG. 4. FIG. 9 is a reference viewillustrating a display screen of an exemplary embodiment of the multidisplay apparatus 100 of FIG. 8.

With reference to FIG. 8, first of all, the sensor 150 senses the firstuser manipulation where a touch input by rotation, that is where aninitial touch is made in the first display 190 a and a drag is made by arotations in one direction, corresponding to a touch & rotate (operationS810). That is, the first display 190 a displays the nonstructured firstscreen, and the sensor 150 senses that there is a touch input by a touch& rotate at a nondecisive location of the first screen.

After the first user manipulation is sensed, a second user manipulationof a touch input by a drag regarding the second display 190 b is sensed(operation S820). Herein, the touch input by the drag may be made by theinput pen 200. In the exemplary embodiment of the present generalinventive concept illustrated in FIG. 9, a drag input is performed inthe second display 190 b using the input pen 200. The drag input definesa drag route on the second display 190 b.

In this case, the controller 130 recognizes that the first usermanipulation is to convert the texture of the line displayed by thesecond user manipulation, and recognizes that the second usermanipulation is an operation to perform an input along the drag route.In addition, the controller 130 combines both user manipulations andrecognizes the input of a line, the texture of which is changed alongthe drag route (operation S830). The line is then displayed according tothe changed texture along the drag route (operation S840).

Herein, texture denotes characteristics of the material being displayed.For example, in the case of an embossing, the texture denotes a casewhere protruded parts and recessed parts exist in turns so that eachpart is shown as an emboss with protruded parts and an engrave withrecessed parts. Another example may be expressions of a touch of a thinbrush or a touch of a thick brush.

As in FIG. 9, when the first user manipulation is a touch input by arotation in one direction (for example, in a clockwise direction), thetexture of the line or dot displayed on the second display 190 b by theinput pen 200 may be changed (operation S840). On the other hand, whenthe first user manipulation is a touch input by a rotation in anotherdirection (for example, a counterclockwise direction), the texture ofthe line or dot displayed by the input pen 200 may return to theoriginal state or be changed to another shape. That is, when there is afirst user manipulation of a rotation in one direction, the line or dotdisplayed on the second display 190 b by the input pen 200 may beexpressed as a watercolor painting brush, and when there is a first usermanipulation of a rotation in another direction, the line or dot mayreturn to the original state, or may be expressed to show a more precisebrush. However, this is merely an exemplary embodiment, and thus morevarious textures may be expressed.

FIG. 10 is a flowchart specifically illustrating a tool menu display ofa multi display apparatus 100 according to an exemplary embodiment ofthe present general inventive concept, and FIG. 11 is a reference viewillustrating a display screen of the multi display apparatus 100 of FIG.10.

In the case of performing an operation of drawing a picture on the multidisplay apparatus 100 through a touch input, or selecting a certain tooland performing an operation, there may be a need to change the settingof the tool. FIGS. 10 and 11 illustrate methods of easily displaying amenu of the tool in such cases.

With reference to FIG. 10, first the sensor 150 senses a first usermanipulation where a touch input by a touch & hold, that is, an initialtouch is made in the first display 190 a and the touch is maintained fora predetermined time and the touch is not removed (operation S1010). Thefirst display 190 a displays the nonstructured first screen, and thecontroller 130 senses that there is a touch input by a touch & hold at anondecisive location of the first screen using the sensor 150 (operationS1010). While the touch & hold is maintained, a second usermanipulation, such as a touch input by a tap on the second display 190b, is sensed (operation S1020). That is, the second display 190 b alsodisplays the nonstructured second screen, and the sensor 150 senses thatthere is a touch input at a nondecisive location of the second screen.Herein, the touch input by the tap may be made by the input pen 200, asillustrated in FIG. 11.

In this case, the controller 130 may display a menu 1110 (operationS1030), which may change the setting of the tool, in response to a touch& hold input performed on the first screen 190 a and a second usermanipulation on the second screen 190 b. Furthermore, when a first usermanipulation by a touch & hold is performed on the first screen 190 awhile the menu 1110 to set the tool is displayed, and there is a seconduser manipulation such as a touch on the second display 190 b, thealready displayed menu 1110, which may change the setting of the tool,disappears from the screen.

The above process is illustrated in FIG. 11. To be more specific, whenthe user performs a touch & hold input on the first display 190 a withtheir hand while drawing a picture on the multi display apparatus 100,and touches the second display 190 b with the input pen 200 (view 1),the menu 1110, which may set the tool corresponding to the input pen200, is displayed on the second display 190 b (view 2). The user mayperform a setting of the tool corresponding to the input pen 200 throughthe menu 1110. For example, if the tool corresponding to the input pen200 is a color pencil, the user may adjust the color and thickness ofthe color pencil. In addition, when the user performs a touch & holdinput on the first display 190 a again with their hand, and then touchesthe second display 190 b with the input pen 200 (view 3), the menu 1110disappears from the second display 190 b (view 4).

As such, it is possible to grant functions to the input pen 200 and usethe functions without complicated option setting operations.

Hereinbelow is an explanation on various operations of the multi displayapparatus 100 which changes a display of the second display 190 baccording to a first user manipulation when an input is sensed throughthe first display 190 a of the multi display apparatus 100.

FIG. 12 is a flowchart illustrating operations of the multi displayapparatus 100 according to an exemplary embodiment of the presentgeneral inventive concept.

With reference to FIG. 12, the sensor 150 of the multi display apparatus100 senses the first user manipulation on the first display 190 a(operation S1210). That is, with the first display 190 a displaying thenonstructured first screen, the user may touch a nondecisive location ofthe first screen and input a first user manipulation. When a touch pointis sensed by the sensor 150, the controller 130 changes the screendisplayed on the second display 190 b according to the sensed first usermanipulation and displays the changed screen (operation S1220). A screenchanging operation may be embodied in various ways according to the typeof the first user manipulation. For example, depending on how the firstuser manipulation is made, an operation of returning to the previousscreen, an operation of converting into the next screen, and anoperation of returning to the main screen may be selectively performed.

In this state, when the user performs a second user manipulation oftouching the second display 190 b, the sensor 150 senses the second usermanipulation (operation S1230), and the controller 130 changes thescreen which has been changed by the first user manipulation and thendisplayed according to the second user manipulation, and displays thechanged screen on the second display 190 b (operation S1240).

Hereinbelow is more detailed explanation on the exemplary embodiment ofthe present general inventive concept illustrated in FIG. 12. FIG. 13 isa flowchart illustrating a specific exemplary embodiment of the multidisplay apparatus 100 of FIG. 12, and FIG. 14 is a reference viewillustrating a display screen of an exemplary embodiment of the multidisplay apparatus 100 of FIG. 13.

With reference to FIG. 13, when the sensor 150 of the multi displayapparatus 100 senses the first user manipulation such as a drag on thefirst display 190 a (operation S1310), the controller 130 changes thescreen displayed on the second display 190 b to the previous operatedscreen according to the sensed first user manipulation and displays thechanged screen (operation S1320). This may correspond for example to ascrolling command, moving sequentially back through pictures which havebeen previously displayed. In this state, when a second usermanipulation such as a tap is sensed in the second display 190 b by thesensor 150 (operation S1330), the controller 130 changes the previouslyoperated screen being displayed on the second display 190 b, the changebeing according to the second user manipulation, and displays thechanged screen (operation S1340).

With reference to FIG. 14, a picture is displayed on the second display190 b (view 1). Inputs may be performed on this picture with a userobject 50 such as an input pen 200. Such inputs may perform operationssuch as modifying the picture. When a touch input by a drag is performedin a downward direction on the first display 190 a (view 2), a previouspicture from among the pictures stored in the second display 190 b isdisplayed (view 3). In addition, a new input may be performed with aninput pen 200 on the second display 190 b regarding the displayedpicture, and perform an operation such as modifying the picture.

Furthermore, when a touch input by a drag in an upwards direction isperformed on the first display 190 a (view 4), a picture stored afterthe present picture is displayed on the second display 190 b (view 5).In the exemplary embodiment of the present general inventive conceptillustrated in FIG. 5, such an upward drag on the first display 190 areturns the second display 190 b to the original picture illustrated inview 1. There may be various touch input directions by the dragperformed by the first display 190 a. For example, as an alternativeconfiguration, when there is a touch input by a drag in the leftdirection (not illustrated) on the first display 190 a, the previouspicture from among the pictures stored in the second display 190 b isdisplayed, and when there is a touch input by a drag in the rightdirection (not illustrated) on the first display 190 a, the pictureafter the picture displayed on the second display 190 b from among thestored pictures may be displayed.

FIG. 15 is a flowchart illustrating another exemplary embodiment of themulti display apparatus 100 of FIG. 12, and FIG. 16 is a reference viewillustrating a display screen of an exemplary embodiment of the multidisplay apparatus 100 of FIG. 15.

With reference to FIG. 15, when the sensor 150 of the multi displayapparatus 100 first senses the first user manipulation performing atouch input by a pinch in or pinch out at a certain point on the firstdisplay 190 a (operation S1510), the controller 130 reduces or expandsthe screen displayed on the second display 190 b according to the sensedfirst user manipulation and displays the reduced or expanded screen(operation S1520). More specifically, when the first user manipulationis a pinch out input, the controller 130 may expand the screen displayedon the second display 190 b according to the direction and length of thedrag made by the pinch out input and displays the expanded screen, andwhen the first user manipulation is a pinch in input, the controller 130may reduce the screen displayed on the second display 190 b according tothe direction and length of the drag made by the pinch out input anddisplays the reduced screen. In addition, the controller 130 senses thesecond user manipulation performing a touch input by a tap on the seconddisplay 190 b (operation S1530), and in this case, the controller 130changes the reduced or expanded screen displayed on the second display190 b according to the second user manipulation result and displays thechanged screen (operation S1540).

In the example illustrated in FIG. 16, when a touch input by a pinch outis performed on the first display 190 a (view 1), for example in ahorizontal direction, the picture displayed on the second display 190 bis expanded to correspond to the direction and length of the drag madeby the pinch out and is displayed (view 2). In addition, the userbecomes able to perform a new input on the second display 190 bregarding the expanded and displayed picture with the input pen 200,such as performing an operation of modifying the picture. On the otherhand, when a touch input by a pinch in is performed on the first display190 a (view 3), the picture displayed on the second display 190 b isreduced and then displayed (view 4).

FIG. 17 is a flowchart specifically illustrating another exemplaryembodiment of the multi display apparatus 100 of FIG. 12, and FIG. 18 isa reference view illustrating a display screen of an exemplaryembodiment of the multi display apparatus 100 of FIG. 17.

With reference to FIG. 17, when the sensor 150 of the multi displayapparatus 100 first senses the first user manipulation which performs atouch input by a touch & rotate on the first display 190 a (operationS1710), the controller 130 rotates the screen displayed on the seconddisplay 190 b according to the sensed first user manipulation anddisplays the rotated screen (operation S1720). More specifically, whenthe first user manipulation is a touch & rotate input, the controller130 controls so that the screen displayed on the second display 190 b isrotated and displayed according to the rotate direction and length basedon the point of the touch. In addition, the controller 130 senses thesecond user manipulation such as performing a touch input by a tap bythe second display 190 b (operation S1730), and in this case, thecontroller 130 changes the rotated screen on the second display 190 baccording to the second user manipulation result and displays thechanged screen (operation S1740).

In the example of FIG. 18, when a touch & rotate input is performed onthe first display 190 a, or when there is a rotate input based on thepredetermined location at the touch point (views 1 and 3, respectivelyillustrating clockwise and counterclockwise rotation), the picturedisplayed on the second display 190 b rotates corresponding to therotation direction and length of the touch & rotate input based on thepoint corresponding to the touch point, and is displayed on the seconddisplay 190 b (views 2 and 4, respectively corresponding to the rotationillustrated in views 1 and 3). Regarding the rotated and displayedpicture, the user may perform a new input with an input pen 200 on thesecond display 190 b, becoming able to perform operations such aszooming in on or modifying the picture. In this case, since the pictureprior to rotation is displayed on the first display 190 a, the userbecomes able to perform the desired operation on the picture rotated anddisplayed on the second display 190 b while checking the shape and colorof the entire picture. Of course, the picture displayed on the firstdisplay 190 a may also be rotated as well, depending on theconfiguration of the particular embodiment of the present generalinventive concept.

Hereinbelow is explanation of an exemplary embodiment of extractingcontents from the first display 190 a by the first user manipulation anddisplaying the extracted contents on the second display 190 b by thesecond user manipulation, according to the present general inventiveconcept.

FIG. 19 is a flowchart illustrating operations of a multi displayapparatus 100 according to an exemplary embodiment of the presentgeneral inventive concept.

With reference to FIG. 19, the sensor 150 of the multi display apparatus100 first senses the first user manipulation on the first display 190 a(operation S1910). That is, the first display 190 a displays thenonstructured first screen, and the sensor senses that there is a firstuser manipulation at a nondecisive location of the first screen.

The controller 130 extracts contents displayed on the first display 190a according to the sensed first user manipulation (operation S1920). Inaddition, the sensor 150 senses a second user manipulation on the seconddisplay 190 a (operation S1930). That is, the second display 190 bdisplays the nonstructured second screen, and the sensor 150 senses thatthere is a second user manipulation at a nondecisive location of thesecond screen. The controller 130 displays the screen on the seconddisplay 190 b corresponding to the extracted contents from the firstdisplay 190 a according to the second user manipulation (operationS1940).

Hereinbelow is specific explanation on the exemplary embodiment of thepresent general inventive concept illustrated in FIG. 19.

FIG. 20 is a flowchart illustrating an exemplary embodiment of the multidisplay apparatus 100 of FIG. 19, and FIGS. 21 to 23 are reference viewsof a display screen of an exemplary embodiment of the multi displayapparatus 100 of FIG. 20.

With reference to FIG. 20, first the sensor 150 senses a first usermanipulation where a touch input by a touch & hold, that is, an initialtouch is made on the first display 190 a, and the touch is maintainedfor a predetermined time (operation S2010).

When the first user manipulation is sensed, the controller 130 capturesthe screen displayed on the first display 190 a (operation S2020).

After the first user manipulation is sensed, the sensor 150 senses asecond user manipulation where a touch input by a double tap or drag onthe second display 190 b is made (operation S2030). Herein, a touchinput by the double tap or drag may be made by an input pen 200.

In this case, the controller 130 displays the screen captured from thefirst display 190 a or a screen related thereto on the second display190 b where a touch is made by the second user manipulation (operationS2040).

In the exemplary embodiment illustrated in FIG. 21, a screen isdisplayed on the first display 190 a and a second screen is displayed onthe second display 190 b (view 1). When a point is touched on the firstdisplay 190 a and a predetermined time has passed, such that a touch &hold input is detected, the screen of the first display 190 a iscaptured (view 2). When a touch input by a double tap is made on thesecond display 190 b, for example with the input pen 200 (view 3), amemo 2110 including an image of the screen captured at the touch pointof the first display 190 a is inserted and displayed on the seconddisplay 190 b (view 4). The user may perform operations such asperforming an input of writing, storing, or uploading necessary contentstogether with the captured screen in the memo 2110, for example with theinput pen 200.

In the exemplary embodiment illustrated in FIG. 22, a screen isdisplayed on the first display 190 a and a second screen is displayed onthe second display 190 b (view 1). When a point is touched on the firstdisplay 190 a and a predetermined time has passed, such that a touch &hold input is detected (view 2), the screen of the first display 190 ais captured. When a touch input by a drag is made in the second display190 b with the input pen 200 (view 3), the image regarding the screencaptured at the touch point is uploaded to the message input window L2210 (view 4).

In the exemplary embodiment illustrated in FIG. 23, a screen isdisplayed on the first display 190 a and a second screen is displayed onthe second display 190 b (view 1). When a point is touched in the firstdisplay 190 a and a predetermined time has passed, such that a touch &hold input is detected (view 2), the screen of the first display 190 ais captured. When a touch input by a drag is made on the second display190 b with an input pen 200 (view 3), an image similar to the capturedimage is searched and a search result screen is displayed on the seconddisplay 190 b (view 4).

FIG. 24 is a flowchart specifically illustrating another exemplaryembodiment of the multi display apparatus 100 of FIG. 19, and FIG. 25 isa reference view illustrating a display screen of an exemplaryembodiment of the multi display apparatus 100 of FIG. 24. Morespecifically, FIGS. 24 and 25 illustrate an exemplary embodiment of thepresent general inventive concept in which the first user manipulationon the first display 190 a and the second user manipulation on thesecond display 190 b are followed by a third user manipulation.

With reference to FIG. 24, first the sensor 150 senses each of a firstuser manipulation and a second user manipulation. Each of the first andsecond user manipulations may correspond to a touch input on the firstdisplay 190 a and the second display 190 b by a touch & hold, that is,where an initial touch is made on each of the first display 190 a andthe second display 190 b, and each touch is maintained for apredetermined time (operation S2410).

When the first and second user manipulations are sensed, the controller130 captures each screen displayed on the first display 190 a and thesecond display 190 b (operation S2420). When the first and second usermanipulations are sensed, the controller 130 may perform a captureoperation corresponding to each of the first and second usermanipulations independently. In this case, the second display 190 b maydisplay the nonstructured second screen, and the second usermanipulation may be made in a nondecisive location of the second screen.Herein, the screens displayed in the first display 190 a and the seconddisplay 190 b are both captured as part of operation S2420.

After the first and second user manipulations are sensed, the sensor 150senses a third user manipulation where a touch input by a double tapregarding the second display 190 b is made (operation S2430). Herein, atouch input by the double tap or drag may be made by the input pen 200.

When there exists a third user manipulation unlike in the aforementionedexemplary embodiment of the present general inventive concept, thecontroller 130 displays the two captured screens on a point on thesecond display 190 b where the third user manipulation is made(operation S2440).

In the exemplary embodiment illustrated in FIG. 25, a screen isdisplayed on the first display 190 a and a second screen is displayed onthe second display 190 b (view 1). When a point is touched on the firstdisplay 190 a and a predetermined time passed, such that a touch & holdinput is detected, the screen of the first display 190 a is captured.Independently from this, when a certain point is touched in the seconddisplay 190 b and a predetermined time passed, the screen of the seconddisplay 190 b is captured (view 2). When a touch input by a double tapis made in the second display 190 b with the input pen 200 (view 3), amemo 2510 is inserted at the touch point, including an imagecorresponding to each screen captured (view 4). The user may performoperations of performing inputting necessary contents together with thecapture screen, or storing or uploading the contents to a sns (SocialNetwork Service) together with the capture screen with the input pen200.

When there is a third user manipulation after the aforementioned firstuser manipulation, the third user manipulation may be differentiatedfrom the second user manipulation. That is, since there is a possibilitythat a touch corresponding to the second user manipulation may bedetermined as the third user manipulation, when a touch input regardingthe second display 190 b is maintained for a predetermined time, thetouch input may be determined as the touch for the second usermanipulation, and when the touch input ends before the predeterminedtime passes, the touch input may be determined as an input for the thirduser manipulation.

When the first user manipulation and the second user manipulation areinput at the same time, unlike in the exemplary embodiment of thepresent general inventive concept illustrated in FIG. 25, the usermanipulations may be recognized as a single user manipulation. That is,when a touch input by a touch & hold is made on the second display 190 bat the same time as the touch & hold input is made on the first display190 a, all the display screens having a touch & hold by the usermanipulation may be captured and the aforementioned operations may beperformed.

Meanwhile, in the aforementioned exemplary embodiments of the presentgeneral inventive concept illustrated in FIGS. 2 to 25, the controller130 may connect the sensed first user manipulation and the sensed seconduser manipulation and recognize them as one user gesture, and maydisplay the execution screen corresponding to the user gesture on atleast one of the first display 190 a and the second display 190 b.

That is, in this case, the controller 130 senses the first usermanipulation and the second user manipulation, but the controller 130recognizes these as a single manipulation and performs operationscorresponding to this single manipulation, instead of performingoperations corresponding to each of the first and second manipulations.In addition, the controller 130 may perform each of the operationscorresponding to the separate first and second user manipulations whenrecognizing the first and second user manipulations as a singlemanipulation.

For example, in an exemplary embodiment of the present general inventiveconcept where the first user manipulation changes the functionscorresponding to the second user manipulation, the controller 130 maynot change the functions corresponding to the second user manipulationsolely in response to the first user manipulation. Instead, when thereis a second user manipulation after the first user manipulation, thecontroller 130 may then change the functions corresponding to the secondmanipulation, and operations according to the changed functions.

As such, the controller 130 may recognize one user manipulationaccording to the result of combining the first user manipulation, seconduser manipulation, and third user manipulation, and may control toperform operations corresponding to the user manipulation according tothe combination results.

Hereinbelow is explanation on the detailed configuration of the multidisplay apparatus 100 which may be embodied differently according tovarious exemplary embodiments of the present general inventive concept.

FIG. 26 is a block diagram illustrating a configuration of a multidisplay apparatus 100 according an exemplary embodiment of the presentgeneral inventive concept.

With reference to FIG. 26, the multi display apparatus 100 of theexemplary embodiment of the present general inventive concept includes acommunication unit 110, multimedia unit 120, controller 130, imagingunit 140, sensor 150, input/output unit 160, storage unit 170, powerunit 180, the first display 190 a, and the second display 190 b.

The communication unit 110 is configured to transmit and receiveinformation to and from an external apparatus (not illustrated) usingvarious wired/wireless communication methods. Herein, the externalapparatus may include at least one of another multi display apparatus,mobile phone, smart phone, tablet PC (Personal Computer), computerserver, and digital TV. These apparatuses may be connected to the multidisplay apparatus 100 through a communication network (not illustrated).

The communication unit 110 may include a connector 114 which includes atleast one of wireless communication modules such as for example acellular communication module 111, wireless LAN module 112, shortdistance communication module 113, GPS communication module 115, andbroadcast communication module 116. The communication unit may alsoinclude wired communication modules (not illustrated), such as forexample an HDMI (High-Definition Multimedia Interface), USB (UniversalSerial Bus), IEEE (Institute of Electrical and Electronics Engineers)1394, and so on.

The cellular communication module 111 uses wireless access technologyaccording to a cellular communication protocol by a control of thecontroller 130 so that the multi display apparatus 100 may be connectedto an external apparatus (not illustrated), such as a base station ofthe cellular system, through at least one or a plurality of antennas(not illustrated).

In addition, the cellular communication module 111 transmits/receiveswireless signals containing voice calls, video calls, Short MessagingServices (SMS), messages or multimedia (Multimedia Messaging Service:MMS) messages with compatible apparatuses such as a mobile phone havingtelephone numbers input into the multi display apparatus 100, a smartphone, a tablet PC, or other apparatuses.

The wireless LAN module 112 is configured to access a wireless AP(access point, not illustrated) existing within a predetermined rangeand of be connected to the internet according to the control of thecontroller 130. The wireless LAN module 112 may support for example thewireless LAN standard (IEEE802.11x) of the IEEE.

The short distance communication module 113 is configured to perform ashort distance communication wirelessly between the multi displayapparatus 100 and the external apparatus according to the control of thecontroller 130. The short distance communication module 113 may includefor example at least one of a Bluetooth module, an IrDA (infrared dataassociation) module, a NFC (Near Field Communication) module, a Wi-Fimodule, and a Zigbee module.

As such, the communicator 110 may be embodied by the aforementionedvarious short distance communication methods. Other communicationtechnologies not mentioned herein may be adopted when necessary,according to the configuration of the particular embodiment of thepresent general inventive concept.

The connector 114 is configured to provide an interface with variousapparatuses such as for example USB 2.0, USB 3.0, HDMI, IEEE 1394, andso on.

The connector 114 may be used as an interface to connect the multidisplay apparatus 100 and the external apparatus or a power source (notillustrated). By the control of the controller 130, data stored in thestorage unit 170 of the multi display apparatus 100 may be transmittedto an external apparatus (not illustrated) or data may be received fromthe external apparatus. Through for example a wired cable connected tothe connector 114, power may be input from a power source (notillustrated) or may be charged in a battery (not illustrated).

The GPS module 115 may receive electric waves from a plurality of GPSsatellites (not illustrated) in the Earth orbit, and may calculate thelocation of the multi display apparatus 100 using Time of Arrival andGPS parameters from the GPS satellites (not illustrated) to the multidisplay apparatus 100.

The broadcast communication module 116 may receive broadcast signals(for example, TV broadcast signals, radio broadcast signals or databroadcast signals) and broadcast additional information (for example,EPS (Electric Program Guide) or ESG (Electric Service Guide))transmitted from a broadcasting station through broadcast communicationantennas (not illustrated) according to a control by the controller 130.

The multimedia unit 120 is a configurative element to play videocontents, audio contents, and other various multimedia contents. For thepurposes of this application, “playing” contents denotes executing anddisplaying the contents for a user of the multi display apparatus 100.It may also denote playing or replaying the contents. The multimediaunit 120 processes multimedia contents using a parser or codec etc. toplay contents. The multimedia unit 120 may include for example an audiomodule 121 and a video module 122.

The audio module 121 may play digital audio files (for example, fileshaving mp3, wma, ogg or way as file extensions) stored or receivedaccording to the control of the controller 130. The video module 123 mayplay digital video files (for example, files having mpeg, mpg, mp4, avi,mov, or mkv as file extensions) stored or received according to thecontrol of the controller 130.

The video module 122 supports codecs of various forms to play thedigital video files. That is, the video module 122 plays a video file bythe codec prestored in a format suitable to the format of the video fileto be played. In addition, the audio module 122 or video module 123 ofthe multimedia unit 120 may be included in the controller 130.

The controller 130 is configured to control the communication unit 110,multimedia unit 120, imaging unit 140, sensor 150, input/output unit160, storage unit 170, power unit 180, first display 190 a, and seconddisplay 190 b. As illustrated in FIG. 27, the controller 130 may includea CPU (Central Processing Unit) 131 configured to provide a clock and totransmit a control signal, a memory such as a RAM (Random Access Memory)135 and ROM (Read Only Memory) 137 configured to temporarily orsemi-permanently store a process, a GPU (Graphics Processing Unit) 133for graphic processing, and a system bus 139 configured to transmit databetween the memory, CPU 131, and GPU 133. In addition, the controller130 also includes an operating system (OS) 420 (illustrated in FIG. 28)to drive the hardware configuration, and an application configured toprovide the user interface on the OS 420 and delivering it to theframework. These components of the controller 130 may be explained infurther detail hereinbelow.

The imaging unit 140 may include at least one of the first camera 141and second camera 142, as illustrated in FIG. 26. Although FIG. 26 onlyillustrates the first camera 141 and the second camera 142, more camerasmay be added according to exemplary embodiments of the present generalinventive concept.

Each camera includes a shutter (not illustrated), lens (notillustrated), aperture (not illustrated) and CCD (Charge Coupled Device)image sensor (not illustrated) and ADC (Analog/Digital Converter). Thelens receives light from an external source (not illustrated) andprocesses the images. The shutter is an apparatus configured to adjustthe volume of light entering the camera, together with the aperture. Theaperture adjusts the quantity of light (light quantity) entering thecamera according to a degree by which the aperture is opened or closed.The CCD image sensor accumulates the quantity of light entering throughthe lens, obtains an image, and outputs the image in accordance with avertical sync signal according to the accumulated quantity of light. TheCCD image sensor obtains the image by converting the received light intoelectrical signals. In order to obtain a color image using the CCD imagesensor, a color filter is needed, and a filter called CFA (Color FilterArray) may be adopted. The CFA passes only light that represents justone color per each pixel, has a regular arrangement structure, and hasvarious forms according to the arrangement structure. The ADC convertsthe analog image signals output from the CCD image sensor into digitalsignals. This is a mere example, and configurations of each camera maybe changed in various ways. For example, each camera may photograph animage using a CMOS (Complementary Metal Oxide Semiconductor) imagesensor instead of a CCD image sensor.

The first camera 141 and second camera 142 may be provided in a housingof the multi display apparatus 100 or may be connected to the multidisplay apparatus 100 using additional connection means. At least one ofthe first camera 141 and second camera 142 may include a subsidiarylight source (for example, a flash, not illustrated) which provides aquantity of light needed to properly obtain an image.

In an exemplary embodiment of the present general inventive concept, thefirst camera 141 may be arranged on a front surface of the multi displayapparatus 100, and the second camera 142 may be arranged in the rearsurface of the multi display apparatus 100. In another exemplaryembodiment of the present general inventive concept, the first camera141 and second camera 142 may be arranged adjacent to each other (forexample, so that the distance between the first camera 141 and thesecond camera 142 is bigger than 1 cm but smaller than 8 cm), andphotograph 3-dimensional still images or 3-dimensional videos. Inanother exemplary embodiment of the present general inventive concept,the first camera 141 may be arranged in the first display 190 a, and thesecond camera 142 may be arranged in the second display 190 b.

The imaging unit 140 may detect a movement or shape of a user object 50through at least one of the first camera 141 and second camera 142, andtransmit the detected movement or shape to the controller 130 as aninput to execute or control an application. In an exemplary embodimentof the present general inventive concept, movement of the user object 50denotes the movement of the user's hand sensed through the first camera141 or second camera 142. The shape of the user object 50 may denote forexample the user's facial shape or orientation of fingers of their hand,sensed through the first camera 141 or second camera 142.

In another exemplary embodiment, the multi display apparatus 100 maydetect the user's movement using another means such as an infraredsensor (not illustrated) and may control or execute the application inresponse to the detected movement.

The sensor 150 is a configurative element to sense various changes ofstate, such as for example the user's touch on the multi displayapparatus 100, the user's movement, or movement of the multi displayapparatus 100 itself. In the aforementioned exemplary embodiment of thepresent general inventive concept, a touch sensor 151 or proximitysensor 155 was explained as an example of the sensor 150. Besides theaforementioned exemplary embodiment of the present general inventiveconcept, the sensor 150 may include at least one of a touch sensor 151,a geomagnetic sensor 152, an acceleration sensor 153, a hinge sensor 154and a proximity sensor 155.

The touch sensor 151 is a sensor which may sense contact of the userobject 50 regarding the multi display (dual display) 190 of the multidisplay apparatus 100. That is, the touch sensor 151 denotes a sensorwhich may sense an input of touching the first display 190 a or thesecond display 190 b and selecting an object displayed on the toucheddisplay. The touch sensor 151 may be classified as an electrostatic typeand a piezoelectric type depending on the method of sensing the touch ofthe user. The touch sensor 151 according to an exemplary embodiment ofthe present general inventive concept may be embodied as any one of thetwo types. The touch sensor 151 may be included in the multi display 190together with a display panel 235 (illustrated in FIG. 29).

The geomagnetic sensor 152 is a sensor which may detect the flow of amagnetic field and detect the azimuth of the magnetic field. Thegeomagnetic sensor 152 may detect the azimuth coordinates of the multidisplay apparatus 100, and may detect the direction of the multi displayapparatus 100 when it is placed based on the azimuth coordinates. Thedetected direction may be recognized by the control input correspondingthereto and the controller 130 may perform the corresponding output.

The acceleration sensor 153 is a sensor used to detect the accelerationof a moving object (dynamic acceleration), but is also utilized todetect the acceleration due to gravity as well. The acceleration sensor153 may set virtual x, y, z axes on the multi display apparatus 100,detect the acceleration of the multi display apparatus 100, and detectthe value of acceleration due to gravity, which changes according to aninclined degree of each axis.

The hinge sensor 156 may detect angles or movement of the hinge 185(illustrated in FIG. 1). The proximity sensor 155 may detect whether ornot a user object 50 approaches the multi display apparatus 100. Theproximity sensor 155 will be explained in further detail below.

Although not illustrated in FIG. 26, the sensor 150 of the multi displayapparatus 100 may further include at least one of a gravity sensor whichmay detect in which direction gravity acts, a gyro sensor which mayrecognize a total of 6 axes due to rotations applied to the accelerationsensor 153, an orientation sensor which may automatically sense a widthand length of contents, such as an image, and align the sensed width andlength, an illuminance sensor which may detect the quantity of lightsurrounding the multi display apparatus 100, a height measuring sensorwhich may measure the air pressure, an RGB sensor which may detect thecolor of objects, a distance measurement sensor which may measure thedistance using an ultrasound wave or infrared ray, and a hall sensorwhich uses changes of voltage according to the intensity of the magneticfield.

The sensor 150 may detect the state of each of the sensors describedabove, generate a signal corresponding to the detection, and transmitthe generated signal to the controller 130. The sensors of the sensor150 described above may be added or deleted according to the performanceand specific configuration of the multi display apparatus 100.

The input/output unit 160 is a configurative element to perform aninput/output using a screen or other external connection ports. Morespecifically, the input/output unit 160 may receive an input signaltransmitted from an input means (not illustrated) such as a mouse,keyboard, joystick connected to the multi display apparatus 100, or froma wireless input means (not illustrated) such as a remote control, etc.,and transmit the received input signal to the controller 130. Otherwise,the input/output unit 160 may output various signals or data generatedby the controller 130 to the external apparatus. The input/output unit160 may include a button 161, a microphone 162, a speaker 163 and avibration motor 164.

The at least one button 161 may be formed in a push type or touch typein a front surface, lateral service, or rear surface of the housing ofthe display apparatus 100, and may include at least one of thepower/lock button, volume adjusting button, menu button, home button,back button and search button.

When the button 161 is pressed, for example by the user, a correspondingcontrol command is generated and transmitted to the controller 130, andthe controller 130 controls operations of the multi display apparatus100 according to the corresponding control command.

The microphone 162 receives voice or sound according to the control ofthe controller 130 and generates a corresponding electrical signal.

The speaker 163 may output sound corresponding to various signals (forexample, a wireless signal, a broadcast signal, a digital audio file, adigital video file, photography, etc.) of the cellular communicationmodule 111, wireless LAN module 112, short distance communication module113, multimedia unit 120 or imaging unit 140 to an outside of the multidisplay apparatus 100.

The speaker 163 may output sound (for example, button manipulating soundor ring back tone corresponding to telephone calls) corresponding to thefunctions performed by the multi display apparatus 100. The speaker 163may be formed as a single speaker or a plurality of speakers, in anappropriate location or locations of the housing of the multi displayapparatus 100. For example, the speaker 164 may be configured to includean inner speaker module (not illustrated) arranged in a locationsuitable to approach the user's ear, and an external speaker module (notillustrated) having a higher output suitable to be used in playingaudio/video files or when viewing broadcast programs, and may bearranged in a suitable location of the housing of the multi displayapparatus 100.

The vibration motor 164 may convert electrical signals into mechanicalvibration according to a control of the controller 130. For example, inthe case of the multi display apparatus 100 in a vibration mode, when avoice call is received from another apparatus (not illustrated), thevibration motor 164 operates. The vibration motor 164 may be formed as asingle motor or a plurality of motors within the housing of the multidisplay apparatus 100. The vibration motor 164 may operate in responseto the user's touch gesture sensed on the first and second display 190a, 190 b, and consecutive movements of the touch sensed on the first andsecond display 190 a, 190 b.

The storage unit 170 is configured to store data.

First of all, the storage unit 170 stores an operating system program tocontrol an operation of the multi display apparatus 100. When the multidisplay apparatus 100 is turned on, the stored operating system is readin the storage unit 170 and compiled to drive each configuration of themulti display apparatus 100.

Secondly, the storage unit 170 is managed by the operating system anduses the resources of the operating system to perform operations of themulti display apparatus 100, and stores an application program providingthe user interface. The application program is read in the storage unit170 by the operating system by the user's execution command, and istransferred to the controller 130 in an executable state to performvarious operations.

Thirdly, the storage unit 170 stores various multimedia data processedby the controller 130, contents data, and data received from externalsources. That is, the storage unit 170 may store signals, information,or data being input/output correspondingly to the operations of thecellular communication module 111, wireless LAN module 112, shortdistance communication module 113, connector 114, GPS module 115,multimedia unit 120, imaging unit 140, sensor 150, input/output unit160, first display 190 a, and second display 190 b according to thecontrol of the controller 130.

The storage unit 170 may be embodied as at least one of a memory card(for example, SD card, memory stick), nonvolatile memory, volatilememory, hard disk drive (HDD) or solid state drive (SSD) which aredetachable/mountable to a ROM, RAM or multimedia apparatus 100.

The power unit 180 supplies power used in the multimedia apparatus 100.The power unit 180 may be embodied as a chargeable battery, and mayfurther include a voltage converter which converts external supply powerand supplies the converted external supply power to a chargeable battery(not illustrated).

The power unit 180 may supply power to the multi display apparatus 100in various modes such as a maximum performance mode, general mode,saving mode, waiting mode etc. according to the power management controlof the controller 130.

The first display 190 a and the second display 190 b are included in thesensor 150 as aforementioned, and output images stored in a frame buffer(described below). The first display 190 a and the second display 190 bmay display multimedia contents, images, videos, and texts etc. by thecontrol of the controller 130.

The first display 190 a and the second display 190 b may be physicallyseparated from each other. The screens displayed on the first display190 a and the second display 190 b may be controlled independently fromeach other. For example, the resolution of the first display 190 a andthe resolution of the second display 190 b may be set independently fromeach other. In addition, expansion, rotation, screen movement, screendivision etc. of the screens displayed on the first display 190 a andthe second display 190 b may be executed independently from each other.

In addition, the first display 190 a and the second display 190 b maydisplay a single screen using a virtual integrated frame buffer.

The first and second display 190 a, 190 b are display apparatuses whichdisplay various applications (for example, telephone calls, datatransmission, broadcasting, camera operation, etc.) which may beexecuted by the controller 130 and which provide user interfacesconfigured to be adaptable thereto. The first display 190 a and thesecond display 190 b may include the aforementioned touch sensor 151, inwhich case at least one touch gesture may be input through a user object50 such as for example the user's body (for example fingers includingthe thumb) or sensible input means (for example, stylus pen)

Such a user interface may include a predetermined touch area, soft keyand soft menu. The first and second display 190 a, 190 b may transmitelectronic signals corresponding to at least one touch gesture inputthrough the user interface to the first display 190 a and the seconddisplay 190 b through an LCD controller (not illustrated). In addition,the first display 190 a and the second display 190 b may sense theconsecutive movement of the touch, and transmit electronic signalscorresponding to the consecutive or nonconsecutive movements of thetouch to the LCD controller. As aforementioned, the touch sensor 151 maybe embodied with a resistive method, capacitive method, infrared method,or acoustic wave method.

The first display 190 a and the second display 190 b may convert thesensed signals regarding the user operations sensed through the touchsensor 151 into digital signals (for example, X and Y coordinates), andtransmit the converted digital signals to the controller 130. Thecontroller 130 uses the received digital signals to perform controloperations corresponding to the user operation input through the firstdisplay 190 a and the second display 190 b. For example, the controller130 may have a soft key displayed on the first display 190 a and/or thesecond display 190 b, and may execute the application corresponding tothe soft key in response to the user operation.

The aforementioned user gesture is not limited to direct contact betweenthe first display 190 a and the second display 190 b, and the userobject 50 (including the user's body and other touchable input means),but includes methods by noncontact as well. The sensitivity of the useroperations detectable in the first display 190 a and the second display190 b may be changed according to the performance and structure of themulti display apparatus 100.

The multi display apparatus 100 is an apparatus configured to executeapplications, widgets and functions which may be stored in the storageunit 170 and may be executed by the controller 130 through a touchscreen 192. A general touch screen 192 may display applications,widgets, functions and graphic objects (that is soft keys or shortcutkeys) that correspond to those groups through a home screen or anapplication menu, and the multi display apparatus 100 executes thecorresponding applications, widgets or functions in response to thedetected user's touch gesture on each displayed graphic object.

Herein, a widget denotes a mini application which may be downloaded bythe user and then used, or may be generated by the user, for example, aweather widget, stock widget, calculator widget, alarm clock widget,dictionary widget etc. A shortcut icon to execute a widget may providesimple dictionary information through the widget applicationcorresponding to the shortcut icon to execute the widget. For example,the icon of the weather widget simply provides the current temperatureand weather symbols, and the widget application executed through a touchof the icon provides more information such as weather per period/region.In the present application, an application includes a widget basedapplication and a non-widget based application.

The first body 2 including the first display 190 a and the second body 4including the second display 190 b may be connected to each other by aconnecting portion such as the hinge 185, and the first body 2 and thesecond body 4 may be folded in or folded out by a certain angle aroundthe hinge 185.

The connecting portion may be embodied as a part of a flexibleconnecting portion (not illustrated), or part of a flexible touch screen(not illustrated), besides the hinge 185.

Hereinbelow is detailed explanation on the configuration of the hardwareof the aforementioned controller 130, with reference to FIG. 27.

FIG. 27 is a block diagram illustrating a hardware configuration of thecontroller 130.

With reference to FIG. 27, the controller 130 of the multi displayapparatus 100 may include a CPU 131, a GPU 133, a RAM 135, a ROM 137 anda system bus 139 in a hardware perspective, and provides an operatingsystem (OS) to drive the hardware configuration in a software (program)perspective, as well as an application to provide the user interface onthe operating system and delivering it to the framework. However,operating systems, frameworks, and applications will be explained infurther detail below.

The CPU 131 controls functions of various blocks while performing datacommunication with each block through the system bus 139, collects theresult of the control, and transmits the control signal to variousperipheral apparatuses connected to the controller 130 based on thecollected result, to perform the function of controlling the peripheralapparatuses. The “peripheral apparatuses” include for example thecommunication unit 110, the multimedia unit 120, the imaging unit 140,the sensor 150, the input/output unit 160, the storage unit 170, and thepower unit 180, as illustrated for example in FIG. 26. In addition, theCPU 131 controls the calculator to read the instruction and argument pereach process in the RAM 135, and to perform calculation.

First, the CPU 131 performs a booting using the booting informationprestored in the ROM 137. That is, when the power of the system isturned on, the CPU 131 reads the command to operate each peripheralapparatus from the ROM 137, and transmits the control signal to eachperipheral apparatus according to the command. In addition, the CPU 131reads the data stored in the storage unit 170 to the RAM 135, andtransmits data which need graphic processing from among the data storedin the RAM 135 to the GPU 133. The CPU 131 receives the data which hasbeen graphically processed by the GPU 133, and transmits the data to theLCD controller (not illustrated) connected to the system bus 139 todisplay the image on the multi display 190.

Herein, the CPU 131 temporarily stores the image data processed by theGPU 133 in the virtual frame buffer allocated to the predetermined areaof the RAM 135. The CPU 131 allocates the area of the virtual framebuffer so as to support the maximum resolution (for example 1024*600) ofthe multi display 190. When there are two displays in the multi display190, the virtual frame buffer area is allocated in a size of 1024*1200.

The CPU 131 inputs the data temporarily stored in the virtual framebutter to the GPU 133 and performs digital signal processing.

The GPU 133 performs graphic processing on the input data under thecontrol of the CPU 131. More specifically, the GPU 133 may generate ascreen including various objects such as an icon, an image, text, etc.using a calculator (not illustrated) and renderer (not illustrated). Thecalculator calculates the characteristics values such as coordinatevalues, shapes, sizes, and colors etc. where each object is to bedisplayed according to the layout of the screen. The rendering generatesscreens of various layouts including the object based on thecharacteristics values calculated in the calculator. The screengenerated in the rendering is transmitted to the first display 190 aand/or the second display 190 b through the bus 139, and is displayedwithin a display area of the first display 190 a and/or the seconddisplay 190 b, or is stored in the storage unit 170.

The CPU 131 may control to display the data graphic processed by the GPU133 through at least one of the first display 190 a and the seconddisplay 190 b, or may input the processed data into a display controller(not illustrated).

The GPU 133 may include a decoder, renderer, and scaler, etc. (notillustrated). Accordingly, the stored contents are decoded, the decodedcontents data is rendered to configure a frame, and the size of theconfigured frame may be scaled to be suitable to the display size by thecontrol of the display controller. If a screen is displayed on one ofthe first display 190 a and the second display 190 b, the scaling ismade to suit the corresponding display size, and if the screen isdisplayed in both of the first display 190 a and the second display 190b, the scaling is made to suit the combination of the two display sizes.The GPU 133 provides the processed frame to the display and displays theprovided processed frame.

The controller 130 may further include an audio processor, an interface,etc. (not illustrated), besides the above. The interface is aconfigurative element to interface with the surrounding configurativeelements.

The audio processor (not illustrated) denotes a configurative elementwhich interfaces with the multimedia unit 120 through the audiointerface (not illustrated) and processes the audio data to provide theprocessed audio data to sound output means such as a speaker. The audioprocessor may decode the audio data stored in the storage unit 170 andthe audio data received through the communicator 110, noise filter thedecoded audio data, and then perform audio signal processing such asamplifying the audio data to an appropriate decibel. In theaforementioned example, when the contents that are played are videocontents, the audio processor may process the audio data demultiplexedfrom the video contents, synchronize the processed audio data with theGPU 133, and provide the result to the speaker 163 to be output.

In ROM 137, a set of commands for system booting etc. is stored. When aturn on command is input and power is supplied, the CPU 131 copies theO/S stored in the storage unit 170 to the RAM 133 according to thecommands stored in the ROM 137, and executes the O/S to boot the system.When the booting is completed, the CPU 131 copies various applicationprograms stored in the storage unit 170 in the RAM 133, and executesapplication programs copied in the RAM 133 to perform variousoperations. As such, the CPU 131 may perform various operationsaccording to execution of the application program stored in the storageunit 170.

As aforementioned, when a touch and other user manipulation is sensed inthe multi display apparatus 100, the controller 130 may determinewhether or not the user manipulation is intended. When it is determinedthat it is an intended user manipulation, the controller 130 readsinformation on the operations corresponding to that user manipulationfrom the storage unit 170, and then performs operations corresponding tothat information. Such operations of the controller 130 may be embodiedby execution of various programs stored in the storage unit 170.

The imaging unit 140 is configured to perform photographing operationsaccording to the user manipulations. The imaging unit 140 may beprovided in plural within the multi display apparatus 100. For example,a first imaging unit (not illustrated) may be provided in the first body2 where the first display 190 a is provided, and a second imaging unit(not illustrated) may be provided in the second body 4 where the seconddisplay 190 b is provided. The controller 130 may store the imagephotographed by the imaging unit 140 or attach the photographed image toan email, document, message, etc. and transmit the attached imageoutside of the multi display apparatus 100 via the communication unit110. In addition, the controller 130 may analyze the image photographedby the imaging unit 140 and recognize the user's motion or gesture, andperform a control operation corresponding to that motion or gesture.

The CPU 131 may be embodied as at least one of a single core processor,multi core processor, triple core processor, and quad core processor. Inthe case of a multi core processor, each processor included in the multicore processor may independently control the first display 190 a andsecond display 190 b.

Hereinbelow is an explanation of a software (program) hierarchyconfiguration of the controller 130.

FIG. 28 is a view illustrating a system hierarchy structure of a multidisplay apparatus 100 according to an exemplary embodiment of thepresent general inventive concept.

With reference to FIG. 28, the multi display apparatus 100 according toan exemplary embodiment of the present general inventive conceptincludes an operating system (OS, 420) configured to drive the hardware,application hierarchies 441-445 managed by the operating system andconfigured to provide user services using resources of the multi displayapparatus 100, and a framework hierarchy 430 relaying the communicationsbetween the operating system and the applications.

The operating system 42 controls the overall operations of the hardware410 and performs functions such as managing the process corresponding toeach application. That is, OS 420 is a hierarchy configured to play thebasic functions such as hardware management, memory, and security etc.The OS 420 includes modules (not illustrated) such as a display driverto drive the multi display, a communication driver to transceive data, acamera driver to drive a camera, an audio driver to drive an audio, anda power manager etc. In addition, an application programming interface(API) library and runtime records (not illustrated) that may be read bya developer (not illustrated) may be included. The OS 420 processes thecalls of the applications, and operates the hardware according to theprocessed results.

There exists a framework hierarchy 430 as a superior hierarchy of OS420. The framework hierarchy 430 performs a role of connecting theapplication hierarchy 440 and the OS hierarchy 420. That is, theframework hierarchy 430 includes a location manager, notificationmanager, and a frame buffer to display an image on the touch screen 192.

There exists an application hierarchy 440 where various functions of themulti display apparatus 100 are embodied in the superior hierarchy ofthe framework hierarchy 430. For example, various application programssuch as call applications 441, multimedia applications 442, cameraapplications 443, browser applications 444, gesture applications 445,etc., may be included.

The applications 441-445 provide the user interface, and receive thecommand from the user and transmit the command received to OS 420through the framework, or requests for resources of the OS 420. The OS420 processes the system call, and manages operation of variousapplications. In addition, the applications 441-445 operate varioushardware, such as the first display 190 a and second display 190 b,according to the system call processing result.

Hereinbelow is explanation of the hardware configuration of the multidisplay 190 according to an exemplary embodiment of the present generalinventive concept.

FIG. 29 is a view illustrating a circuit configuration of an imageoutputter 191 of the first display 190 a or the second display 190 b.Hereinbelow, the explanation will be made based on the first display 190a, but the second display 190 b would include the same configurationsand would operate in the same manner. Furthermore, FIG. 29 illustratesthe circuit configuration of the image outputter 191, whereas FIG. 31illustrates the physical construction of the image outputter 191.

According to FIG. 29, the image outputter 191 of the first display 190 amay include a timing controller 231, a gate driver 232, a data driver233, a voltage driver 234, and a display panel 235.

The timing controller 231 receives a clock signal (DCLK) and horizontalsync signal (Hsync), vertical sync signal (Vsync), etc. and generates agate control signal (injection control signal), and data control signal(data signal), and realigns the received R, G, B values of the image tobe displayed, and provides them to the data driver 233.

The timing controller 231 may generate a gate shift clock (GSC), gateoutput enable (GOE), and gate start pulse (GSP) etc. Herein, a GSC is asignal configured to determine the time when a thin film transistor(TFT) connected to light emitting diodes such as R, G, B (for exampleOrganic Light Emitting Diodes, OLEDs) is turned on/off, a GOE is asignal configured to control the output of the gate driver 232, and aGSP is a signal configured to notify the first driving line of thescreen in one vertical sync signal.

In addition, the timing controller 231 may generate Source SamplingClock (SSC), Source Output Enable (SOE), and Source Start Pulse (SSP)etc. regarding the data control signal. Herein, SSC is used as asampling clock to latch data in the data driver 233, and determines thedriving frequency of the data driver 233. The SOE delivers the datalatched by the SSC to the display panel 235. The SSP is a signalnotifying a latch or sampling start of data during one horizontal syncperiod.

The gate driver 232 is a configuration generating an injection signal,and is connected to the display panel 235 through the injection linesS1, S2, S3, . . . Sn. The gate driver 232 applies gate on/off voltage(Vgh/Vgl, not illustrated) provided from the voltage driver 234 to thedisplay panel 235 according to the gate control signal generated by thetiming controller 231. The gate on voltage (Vgh) is providedconsecutively from gateline 1 (GL1) to gateline N (GLn) for embodimentof unit frame images on the display panel 235. The data driver 233 is aconfiguration of generating a data signal, and is connected to thedisplay panel 235 through data lines D1, D2, D3, . . . Dn. The datadriver 233 finishes scaling according to the data control signalgenerated by the timing controller 233, and inputs the RGB data of theimage into the display panel 235. The data driver 233 converts the RGBimage data provided in serial into parallel in the timing controller231, and converts the digital data into analog voltage to provide theimage data corresponding to one horizontal line portion to the displaypanel 235. This process is made consecutively per horizontal line.

The voltage driver 234 generates and delivers each driving voltage ofthe gate driver 232, data driver 233, and display panel 235 etc. Thatis, the voltage driver 234 may receive prevailing voltage from outsidethe multi display apparatus 100, for example, alternating voltage of110V or 220V, generate and provide source voltage (VDD) necessary in thedisplay panel 235, or provide ground voltage (VSS). In addition, thevoltage driver 234 may generate a gate on voltage (Vgh) and provide itto the gate driver 232. To this end, the driver 234 may include aplurality of voltage driving modules (not illustrated) operatingindependently. Herein, the plurality of voltage driving modules mayoperate to provide different voltages from one another according to thecontrol of the controller 130, and the controller 130 may control thevoltage driver 234 so that a plurality of voltage driving modules couldprovide different driving voltages from one another according to thepredetermined information. For example, each of the plurality of voltagedriving modules may provide a different first voltage according to theinformation predetermined according to the control of the controller 130and a second voltage set in default.

According to an exemplary embodiment of the present general inventiveconcept, the voltage driver 234 may include a plurality of voltagedriving modules corresponding to each area of the display panel 235divided in a plurality of areas. In this case, the controller 130 maycontrol a plurality of voltage driving modules to provide first voltageswhich are different from one another, that is, ELVDD voltages (notillustrated) according to each screen information (or input imageinformation) of the plurality of areas. That is, the controller 130 mayuse the image signal input into the data driver 233 to control the sizeof the ELVDD voltage. Herein, the screen information may be at least oneof the brightness information and gradation information of the inputimage.

The display panel 235 may be provided with a plurality of gatelinesGL1-GLn and datalines D1-Dn configured to intersect one another anddefine pixel areas 236, and in the intersecting pixel areas 236, R, G, Blight emitting diodes such as OLEDs may be formed in the pixel area 236where the gatelines and datalines intersect one another. In addition, inone area of the pixel area 236, more specifically in the corner, aswitching element, that is a TFT, is formed. During the turn onoperation of the TFT, the gradation voltage from the data driver 233 isprovided to each light emitting diode of R, G, B. Herein, R, G, B lightemitting diodes in accordance with the quantity of current providedbased on the gradation voltage. That is, when a large quantity ofcurrent is provided, R, G, B light emitting diodes provide that much ofthe current provided.

FIG. 30 is a view illustrating a circuit structure configuring R, G, andB pixel areas 236 which configure the display panel 235 of FIG. 29.

With reference to FIG. 30, the display panel 235 includes three pixelareas 236 corresponding to R, G, and B colors. The R, G, B pixel areas236 may include a scan signal S1, switches M11, M21, M31 that operate bythe gate on voltage (Vgh, not illustrated), switchers M12, M22, M32 thatoutput current based on the pixel values including the changed highgradation value provided to the datalines D1-Dn, and switchers M13, M23,M33 that adjust the quantity of current provided from the switchers M12,M22, M32 to the R, G, B light emitting diodes according to the controlsignal provided in the timing controller 231. In addition, suchswitchers M13, M23, M33 are connected to the organic light emittingdiodes (OLEDs) to supply current to the OLEDs. Herein, “OLED” denotes adisplay that emits light itself using the principle of electric fieldlight emission when current is applied to fluorescence orphosphorescence organic matter thin film. The anode electrode of eachOLED is connected to the pixel circuit, and the cathode electrode isconnected to the ELVSS. Such an OLED generates light of a predeterminedbrightness in response to the current supplied from the pixel circuit.Herein, the gate electrode of the switchers M11, M21, M31 are connectedto the injection line S1, while the other one of the source electrodeand the drain electrode is connected to the dataline D1. As such, thedisplay panel 235 may be embodied as an AM-OLED (Active Matrix OrganicLight-Emitting Diode) panel. However, the aforementioned exemplaryembodiment is merely one exemplary embodiment of the present generalinventive concept, and a PM OLED (Passive Matrix Organic Light-EmittingDiode), which is a method where one line emits light and drives at thesame time, is obviously not to be excluded.

As such, in the case of embodying the multi display 190 of the multidisplay apparatus 100 with OLEDs, there is no need to provide anadditional light emitting means (backlight), and thus there is anadvantage of making the thickness of the display thinner and simplifyingthe configuration.

However, although OLEDs are described in the above exemplary embodimentof the present general inventive concept, the display may be embodied asvarious display technologies such as Liquid Crystal Display Panel (LCDPanel), Plasma Display Panel (PDP), Vacuum Fluorescent Display (VFD),Field Emission Display (FED), and Electro Luminescence Display (ELD).

FIG. 31 is a cross-sectional view of a first display 190 a including atouch screen 192. Although the configuration illustrated in FIG. 31 isdescribed below with reference to the first display 190 a, the seconddisplay 190 b may have the same construction.

With reference to FIG. 31, the first display 190 a includes an imageoutputter 191 and a touch screen 192.

The image outputter 191 includes the display panel 235, where an uppersubstrate 362 and a lower substrate 361 are disposed, an upperpolarizing plate 300 provided on the upper substrate 362 of the displaypanel 235, and a lower electrode 310 provided in the edge area of theupper polarizing plate 300. In addition, the image outputter 191includes a lower polarizing plate 301 provided on the lower substrate361.

There may be provided a liquid crystal layer 363 between the uppersubstrate 362 and lower substrate 361. The upper substrate 362 and lowersubstrate 361 include a display area to display images and a peripheralarea provided at the edge of the display area. Although not illustrated,in the display area of the lower substrate 361, a thin film transistoris provided in an intersecting area of a plurality of gatelines, and aplurality of gatelines and datalines, as well as a pixel electrodeconnected to the thin film transistor, are provided. There is provided asustained electrode where the pixel electrode and a portion thereofoverlap each other. In addition, in a peripheral area of the lowersubstrate 361, there may be provided a plurality of pads (notillustrated) connected to the gatelines and datalines.

The upper substrate 362 and the lower substrate 361 may be sealed usinga sealing member 364, such as silant. The sealing member 364 iseffective when provided in a peripheral area of the upper substrate 362and lower substrate 361.

The touch screen 192 includes a lower transparent electrode 330connected to the lower electrode 310 and provided on the upperpolarizing plate 300, a touch pad 340 provided on the lower transparentelectrode 330, and a protrusion 350 provided between the uppertransparent electrode 342 and the lower transparent electrode 330.

The touch pad 340 includes a base film 341, the upper transparentelectrode 342 provided on the base film 341, and an upper electrode 343provided on the upper transparent electrode 342.

In the case of pressing the touch pad 340 with a predetermined force,the protrusion 350 of the area where the force is applied contacts thelower transparent electrode 330, and accordingly, the lower transparentelectrode 330 and upper transparent electrode 342 contact each otherelectrically. Through the electrical contact, the current flowingthrough the upper transparent electrode 342 and/or the lower transparentelectrode 330 changes, and this change is sensed through an additionalsensing means (not illustrated). A signal output from the sensing meansis delivered to the controller 130. The controller 130 uses the signalto generate coordinates of the area where the flow of current ischanged, and delivers the generated coordinates to a predetermineddriver (not illustrated). The driver may perform the same operations asthose used in manipulating the image displayed on the display panel 235using the input tool such as a mouse in response to the inputcoordinates.

Meanwhile, due to the characteristics of the multi display apparatus100, it is possible to provide an image frame buffer 134 a-1, 134 b-1,or 134-1 in various methods. Hereinbelow is explanation on image framebutters provided in various methods.

FIG. 32 is a block diagram illustrating a display driver 134 of themulti display apparatus 100 according to an exemplary embodiment of thepresent general inventive concept, and FIG. 33 is a block diagramillustrating a configuration of the display driver 134′ according toanother exemplary embodiment of the present general inventive concept.

According to FIG. 32, the display driver 134 may include a first framebuffer 134 a-1, first display driver 1341-2, second frame buffer 134 b-1and second display driver 134 b-2. That is, each of the displays 190 a,190 b may have an additional frame buffer 134 a-1 134 b-1, and displaydriver 134 a-2, 134-2.

The first frame buffer 134 a-1 is a configuration to buffer an imageframe to be displayed on the display 190 a, and the second frame buffer134 b-1 is a configuration to buffer an image frame to be displayed onthe second display 190 b.

For example, the image frame which has been digital signal processed bythe GPU 133 is stored in a bit map format to the first and second framebuffers 134 a-1, 134 b-1. In this case, the buffering area of each frame134 a-1, 134 b-1 is allocated suitable to the maximum pixel size thatmay be supported in each of first display 190 a and second display 190b. The first display driver 134 a-2 analyzes the image frame stored inthe first frame buffer 134 a-1 and converts the image frame stored inthe first frame buffer 134 a-1 into a first image source signal. Thefirst display driver 134 a-2 provides the first image source signal tothe first display 190 a, to drive the first display 190 a to display theimage frame.

Likewise, the second display driver 134 b-2 analyzes the image framestored in the second frame buffer 134 b-1 and converts the analyzedimage frame into a second image source signal, and provides theconverted second image source signal to the second display 190 b to bedisplayed.

These two frame buffers 134 a-1 and 134 b-1 may be suitable to processthe image frames regarding each of the first display 190 a and thesecond display 190 b in parallel. When a parallel processor (notillustrated) outputs the image signal regarding each of first display190 a and second display 190 b, it is possible to increase the imageoutput efficiency.

However, unlike the above, the first frame buffer 134 a-1 and secondframe buffer 134 b-1 respectively corresponding to each of first display190 a and second display 190 b may be integrated to one integrated framebuffer 134-1 and used instead of being provided separately.

FIG. 33 is a block diagram illustrating a configuration of a multidisplay apparatus 100 controlling operations of each display 190 a, 190b using the integrated frame buffer 134-1. When the frame buffer isembodied as an integrated frame buffer 134-1, the size of the integratedframe buffer 134-1 may be allocated and embodied to be bigger than themaximum resolution of the first display 190 a and the second display 190b.

For example, when each of the first display 190 a and the second display190 b displays the maximum resolution of 1024*800, the integrated framebuffer 134-1 allocates the storage area to be the frame buffer sizecapable of displaying the resolution of 1024*1600. In the first area ofthe integrated frame buffer 134-1, the first image frame where the firstdisplay 190 a is displayed is stored, and in the second area of theintegrated frame buffer 134-1, a second image frame displayed on thesecond display 190 b is stored.

The display driver 134-2 uses the first image frame stored in theintegrated frame buffer 134-1 and the address of the second frame toprovide the first and second image frames to the first display 190 a andthe second display 190 b, to drive each display.

As aforementioned, the sensor 150 of the multi display apparatus 100 mayinclude proximate sensing means, for example, the proximity sensor 155.Hereinbelow is explanation on the configuration and operations of theproximity sensor 155.

FIG. 34 is a view illustrating a proximate touch according to anexemplary embodiment of the present general inventive concept. Theconstruction illustrated in FIG. 34 may be applied to either or both offirst display 190 a and second display 190 b.

According to the exemplary embodiment illustrated in FIG. 34, a user mayinput a control command into the multi display apparatus 100 merely bybringing a user object 50 such as a finger or other object intoproximity of the touch screen 192, without directly touching the touchscreen 192. The multi display apparatus 100 may sense the proximatetouch using the proximity sensor 155. The proximate touch denotesrecognizing as one touch gesture when a movement is recognized within acertain spatial effective recognition range 5, without directly touchingthe touch screen 192.

With reference to FIG. 34, in the upper portion of a display 290, aninfrared source 291 to recognize proximate touches is located, and inthe lower portion of the display 290, an infrared sensor 292 isincluded.

The infrared source 291 irradiates an infrared ray in a surfacedirection of the display 290. More specifically, the infrared source 291is arranged in the lower portion of the display 290 where the image isdisplayed, and may irradiate the infrared ray in the surface directionof the display 290. There exists a certain area which may recognize theapproach of the user object 50 above the surface of the display 290.This area is the effective recognition area 5 in which the proximatetouch may be recognized.

The meaning of the user object 50 is the means to input a command intothe multi display apparatus 100, for example, a body part such as thehand.

When the user object 50 approaches towards inside the effectiverecognition area 5, the sensor 292 senses the infrared reflected by theapproach of the user object 50 and generates the infrared scan image.More specifically, the infrared sensor 292 uses a plurality of infraredsensing elements arranged in an array format to generate an infraredscan image corresponding to the infrared ray reflected by the approachof the user object 50. The multi display apparatus 100 may sense theproximate touch input using the generated infrared scan image.

Hereinbelow is explanation on the exterior configuration of the multidisplay apparatus 100 according to an exemplary embodiment of thepresent general inventive concept with reference to the perspective viewattached.

FIG. 35 is a detailed perspective view of the multi display apparatus100 according to an exemplary embodiment of the present generalinventive concept. Herein, it is illustrated that the multi displayapparatus 100 has two displays 190 a and 190 b, combined by the hinge185.

With reference to FIG. 35, the multi display apparatus 100 includes thefirst body 2 and the second body 4, the first body 2 and the second body4 configured to be connected by the hinge 185 and thus movable relativeto one another. On one surface of the first body 2, there is providedthe first display 190 a, while on one side of the display 190 a, atleast one physical button 161 may be arranged. On one surface of thesecond body 4, the second display 190 b is provided, while on one sideof the second display 190 b, at least one physical button 161′ may bearranged. The physical buttons 161 and 161′ may include at least one ofthe push button and touch button. In an exemplary embodiment of thepresent general inventive concept, the first display 190 a arranged inthe first body 2 having the speaker 163 and microphone 162 may operateas a main screen, while the second display 190 b arranged in the secondbody 4 may operate as a sub screen.

In an exemplary embodiment of the present general inventive concept, thebody 2 has the first camera 141, and the second body 4 has the secondcamera 142.

With the multi display apparatus 100 arranged as illustrated in FIG. 35,each of the displays 190 a and 190 b may display the screen in alandscape mode.

As long as the multi display apparatus 100 has the first body 2 andsecond body 4 connected by the hinge 185 to be relatively moveable, itmay be any apparatus such as a mobile phone, notebook, tablet PC, PMP,etc. The first display 190 a and the second display 190 b are explainedas having the first body 2 and second body 4, respectively, but thefollowing explanation may be applied to any apparatus which provides atouch screen display as only one panel. In addition, at least one of thefunction buttons 161 on the side of each of the displays 190 a, 190 bmay be omitted. Furthermore, although it is explained that a connectionbetween the first body 2 and second body 4 is made by the hinge 185, thehinge 185 may be substituted for any other configurative element as longas the first body 2 and second body 4 are configured to move relative toone another.

Meanwhile, the hinge 185 may include two hinges arranged in the upperand lower sides of the connecting portion of the first body 2 and secondbody 4. In addition, the hinge 185 may include one hinge arranged in theentirety of the connecting portion of the first body 2 and second body4.

The multi display apparatus 100 has the display apparatus including thefirst display 190 a and second display 190 b which are separated fromeach other physically or graphically. The multi display apparatus 100may utilize two displays 190 a and 190 b to support various screen modessuch as illustrated in FIGS. 11 to 15.

FIGS. 35 to 40 are views illustrating various screen modes according tothe relative angle between the first body 2 and the second body 4 of themulti display apparatus 100 according to an exemplary embodiment of thepresent general inventive concept.

The relative angle θ is a rotation angle between the second body 4rotated in a predetermined direction (for example counterclockwisedirection) regarding the first body 2.

More specifically, the relative angle θ may be detected using the hingesensor 154 mounted inside the hinge 185 itself. The hinge sensor 154 mayinclude one of a hall sensor, pressure sensor, induction detectionsensor, electricity contact sensor and optical sensor, and detect themovement of the hinge and relative location to recognize the relativeangle θ.

In addition, besides the hinge sensor 154, the geomagnetic sensor 152and acceleration sensor 153 may detect each location of the first body 2and the second body 4 to recognize the relative angle θ.

FIG. 36 is a perspective view illustrating a folded multi displayapparatus 100 according to an exemplary embodiment of the presentgeneral inventive concept. As illustrated in FIG. 36, with the firstdisplay 190 a on the first body 2 and the second display 190 b on thesecond body 4 facing each other, the first body 2 and the second body 4touch each other. That is, the second display 190 b is located on theopposite side of the first display 190 a. When the user sees the firstdisplay 190 a, the second display 190 b is located at the opposite side,and thus the user cannot see the second display 190 b. As such, the usermay only see one display.

In the configuration illustrated in FIG. 36, the relative angle θ isdefined as 0 degrees. For example, when the relative angle between thefirst body 2 and the second body 4 is 0 to 60 degrees, the multi displayapparatus 100 may recognize as a single display mode. The single displaymode may be used usefully for applications in which only one display isnecessary, for example in a telephone call application, or when themulti display apparatus 100 is in a locked state and is not used. In thesingle display mode, the first display 190 a of the front surfacedisplays the operation screen by at least one application, and thesecond display 190 b on the rear surface may be turned off. Someapplications may use an option menu (not illustrated) to turn on thesecond display 190 b of the rear surface. In the case of the singledisplay mode, the user can see only one display screen, and thuscontroller 130 controls one display only. That is, the controller 130only transmits a control signal regarding the display that the user iscurrently seeing. Additional control signals and data signals are nottransmitted regarding the display which is not controlled by thecontroller 130, thereby saving power consumption.

Herein, the display which is not provided with the control signal anddata signal operates in a sleep mode, and when the signal to be releasedfrom the sleep mode is applied by the controller 130, such as when theuser rotates the multi display apparatus 100 to see the display in thesleep mode, the display may be released from the sleep mode, andtherefore receive control signals from the controller 130.

FIG. 37 illustrates a state where the relative angle θ is orapproximates 180 degrees within a predetermined range, with the firstbody 2 and second body 4 substantially parallel to each other.Hereinbelow this mode is called an expanding mode. For example, when therelative angle between the first body 2 and the second body 4 is in therange of 175 to 185 degrees, the multi display apparatus 100 maydetermine that the first body 2 and the second body 4 are unfolded. Theexpanding mode may display two operation screens regarding twoapplications on each of the displays 190 a and 190 b, or display onescreen regarding one application on both of the displays 190 a and 190b, or display two operation screens regarding one application on the twodisplays 190 a and 190 b. No application may be displayed on either oneof the displays, but a defaulted home screen may be displayed. Theexpanding mode may be useful in applications best performed withmultiple displays or a single large display, such as for example inE-books, video player applications, and web browsing screens.

For example, the controller 130 may display a first web browsing screen(not illustrated) on the first display 190 a, and display a second webbrowsing screen (not illustrated) on the second display 190 b. Herein,the first web browsing screen and second web browsing screen are relatedto each other, and a third web browsing screen (not illustrated) relatedto the second web browsing screen may be displayed on the first display190 a, and a fourth web browsing screen (not illustrated) related to thethird browsing screen may be displayed on the second display 190 b.

The controller 130 may display the web browsing screens alternately onthe first display 190 a and second display 190 b, and may perform screenconversion between the plurality of web browsing screens displayed oneach display 190 a and 190 b, and may display the web browsing screensdisplayed on the first display 190 a and the web browsing screensdisplayed on the second display 190 b in different layouts from eachother.

FIG. 38 illustrates a state of the multi display apparatus 100 in whichthe relative angle θ of the second body 4 to the first body 2 exceeds180 degrees, that is, when the two displays 190 a and 190 b are foldedslightly towards an inside direction, i.e., folded slightly towards eachother. This is referred to herein as a tool kit mode. For example, whenthe relative angle between the first body 2 and second body 4 is 180 to265 degrees, it is recognized as the tool kit mode. The tool kit mode isuseful when the multi display apparatus 100 is used in a similar formatas a notebook computer. For example, various operation environments maybe provided such as displaying an operation screen on the first display190 a and displaying a tool such as a keyboard on the second display 190b.

FIG. 39 illustrates a state where the relative angle θ of the first body2 and second body 4 is less than 180 degrees, that is, when the twodisplays 190 a and 190 b are folded towards an outside, i.e., folded sothat the displays 190 a and 190 b face almost opposite directions. Thismode is referred to herein as a standing mode. For example, when therelative angle between the first body 2 and the second body 4 is between30 to 90 degrees, the multi display apparatus 100 may recognize thestanding mode. Since the standing mode is a structure where the twodisplays 190 a and 190 b are folded towards outside, it is possible tostand the multi display apparatus 100 on the bottom in a triangularshape, i.e., balanced on an edge portion of the first body 2 and thesecond body 4. The standing mode of the multi display apparatus 100 maybe useful when charging, viewing an image (for example, using the multidisplay apparatus 100 as a digital clock or frame), or watching personalbroadcast, film, and video etc. In another exemplary embodiment of thepresent general inventive concept, the standing mode may be applied tothe application which requires cooperation or interaction by two or moreusers, for example, a conference call, a collaborative game, etc. Someapplications display operations screens on only the first display 190 ain the standing mode, and may turn off the second display 190 b. Someapplications may turn on the second display 190 b using the option menu.

FIG. 40 is another exemplary embodiment of the standing mode illustratedin FIG. 39, referred to herein as length view mode where the multidisplay apparatus 100 is placed in a standing manner so that a portionof the hinge 185 touches the ground. When it is determined that therelative angle θ between the first body 2 and the second body 4 is 30 to90 degrees and that the multi display apparatus 100 is placed in astanding manner by the acceleration sensor 153, the multi displayapparatus 100 may recognize the length view mode.

More specifically, the acceleration sensor 153 may sense rotation of themulti display apparatus 100. The acceleration sensor 153 may sense theconversion between the length view mode where the first display 190 aand the second display 190 b are placed on the left or right side, asillustrated in FIG. 40, and a width view mode where the first display190 a and the second display 190 b are placed on an upper or lower side,as illustrated in FIG. 39.

The length view mode may be applied to an application where differentimages need to be provided to two or more users, for example, conferencecall and multi video player etc.

FIGS. 41 and 42 are views illustrating an arrangement of two cameras 141and 142 of the multi display apparatus 100 according to an exemplaryembodiment of the present general inventive concept. FIGS. 41 and 42 areviews of the multi display 100 apparatus seen from the direction of bothdisplays 190 a and 190 b while in the expanding mode. These are views toillustrate the arrangement of the cameras 141 and 142, and thus otherelements and configurations of the multi display apparatus 100 areomitted for convenience of explanation.

With reference to FIG. 41, the multi display apparatus 100 includes afirst body 2 and second body 4, the first body 2 including the firstdisplay 190 a and the second body 4 including the second display 190 b.The first body 2 and second body 4 are connected by the hinge 185 sothat they are moveable relative to one another. The first camera 141 maybe arranged on the first body 2 in a central portion of an edge areaopposite to the hinge 185. Similarly, the second camera 142 may bearranged on the second body 4 in a central portion of an edge areaopposite to the hinge 185.

Next, an alternate arrangement of first camera 141′ and second camera142′ is explained with reference to FIG. 42. Similarly to FIG. 41, theexplanation is made with the multi display apparatus 100 in theexpanding mode. The first camera 141′ may be arranged on a centralsurface of the left edge area of the first body 2, and the second camera142′ may be arranged on a central surface of the left edge of the secondbody 4. In another exemplary embodiment of the present general inventiveconcept, each of the first camera 141′ and the second camera 142′ may bearranged in a central surface of the right edge area of the first andsecond body 2, 4, respectively. In another exemplary embodiment of thepresent general inventive concept, the first and second camera may bearranged in a corner area of the first body 2 and the second body 4,respectively. The multi display apparatus 100 with the first camera 141′and the second camera 142′ arranged as in FIG. 42 may not only be usedin the width view mode as aforementioned but also in the length viewmode.

Hereinbelow is explanation on a multi display method according tovarious exemplary embodiments of the present general inventive concept.

FIGS. 43 to 47 are flowcharts illustrating a multi display methodaccording to various exemplary embodiments of the present generalinventive concept.

The operations of the multi display method already explained above willbe omitted.

The multi display method as in FIG. 43 may be performed in a multidisplay apparatus 100 including a first body 2 where a first display 190a is arranged, a second body 4 where a second display 190 b is arranged,and a hinge 185 which connects the first body 2 and the second body 4and supports the first body 2 and the second body 4 so that they arerotatable relative to one another.

More specifically, with the multi display apparatus 100 displaying theunstructured first screen and second screen, through the first display190 a and the second display 190 b, respectively, the user may performuser operations for each screen. Accordingly, when a first usermanipulation regarding a nondecisive location of the first screen issensed (operation S4310), and a second user manipulation regarding anondecisive location of the second screen is sensed (operation S4320),the sensed first user manipulation and second user manipulation arecombined, and an operation corresponding to that combination results ifperformed (operation S4330).

With reference to FIG. 44, the multi display apparatus 100 according toan exemplary embodiment of the present general inventive concept mayinclude sensing the first user manipulation (operation S4410), changingthe functions matching the second user manipulation according to thefirst user manipulation (operation S4420), sensing a second usermanipulation (operation S4430), and performing an operation according tothe changed functions when the second user manipulation is sensed(operation S4440).

Herein, the second user manipulation may be a line input, and when thefirst user manipulation is a pinch out input, the changing the function(operation S4420) may increase the thickness of the input line, and whenthe first user manipulation is a pinch in input, the changing thefunction (operation S4420) may reduce the thickness of the input line,and when the first user manipulation is a touch & rotate input, thechanging the function (operation S4420) may change the texture of theline.

In addition, in performing the aforementioned operations (operationS4330) in the exemplary embodiment illustrated in FIG. 43, if the firstuser manipulation is a touch & hold input, and when the second usermanipulation by the touch input is sensed together with the first usermanipulation, a menu corresponding to the second user manipulation maybe displayed.

With reference to FIG. 45, the multi display method according to anexemplary embodiment of the present general inventive concept includessensing a first user manipulation (operation S4510), changing the screendisplayed on the second display according to the first user manipulationwhen the first user manipulation is sensed (operation S4520), sensing asecond user manipulation (operation S4530), and changing the screenwhich has been changed by the first manipulation and then displayed whenthe second user manipulation is sensed (operation S4540).

Herein, changing the screen (operation S4520) may expand the screendisplayed on the second display when the first user manipulation is apinch out input, reduce the screen displayed on the second display 190 bwhen the first user manipulation is a pinch in input, and rotate thescreen displayed on the second display when the first user manipulationis a touch & rotate input.

With reference to FIG. 46, the multi display method according to anexemplary embodiment of the present general inventive concept mayinclude sensing a first user manipulation (operation S4610), extractingcontents displayed on the first display 190 a when the first usermanipulation is sensed (operation S4620), sensing a second usermanipulation (operation S4630), and displaying a screen corresponding tothe extracted contents on the second display 190 b when the second usermanipulation is sensed (operation S4640).

Herein, the extracting (operation S4620) may capture the screendisplayed on the first display 190 a according to the first usermanipulation, and displaying the screen (operation S4640) may displaythe captured screen in the location where the second user manipulationis made on the second display 190 b.

With reference to FIG. 47, the multi display method of the multi displayapparatus 100 according to an exemplary embodiment of the presentgeneral inventive concept includes sensing a first user manipulationregarding the nondecisive location of the first screen when each of thefirst display 190 a and second display 190 b displays the unstructuredfirst screen and second screen, respectively (operation S4710), sensingthe second user manipulation regarding the nondecisive location of thesecond screen (operation S4720), and connecting the sensed first usermanipulation and second user manipulation to recognize as one usergesture, and displaying the execution screen corresponding to the usergesture at one of the first display 190 a and the second display 190 b(operation S4730).

In the aforementioned multi display method according to variousexemplary embodiments of the present general inventive concept, thefirst user manipulation may be a proximate input or touch input to thefirst display 190 a using the user's body, and the second usermanipulation may be a proximate or touch input to the second display 190b using an input pen 200.

Meanwhile, the aforementioned various multi display methods wereexplained to recognize the nonstructured user manipulation made for eachof the plurality of displays in the multi display apparatus 100 andperform operations accordingly. However, the present general inventiveconcept is not limited thereto, but rather, the nonstructured usermanipulation made for each of the plurality of displays may berecognized as one user manipulation and an operation corresponding tothe one user manipulation may be performed.

For example, when the user performs a drag operation from a point on thefirst display 190 a to the second display 190 b, a user manipulation issensed in both the first display 190 a and second display 190 b,respectively. When the point where the user manipulation is sensed inthe first display 190 a and the second display 190 b is within thepredetermined time range and when the dragging track on the firstdisplay 190 a is continuous with the dragging track on the seconddisplay 190 b, the controller 130 recognizes those user manipulations asone dragging operation. Accordingly, an operation matching the draggingmanipulation is performed. For example, the controller 130 may performvarious operations such as for example an operation of expanding thefirst screen displayed on the first display 190 a where the initialdragging began on the second display 190 b, an operation of dividing onescreen and displaying the divided screen on one or both of the firstdisplay 190 a and the second display 190 b, an operation of displayingone common screen on each of the first display 190 a and the seconddisplay 190 b, an operation of simultaneously activating or inactivatingthe first display 190 a and the second display 190 b, and an operationof turning on or off power of the multi display apparatus 100.

The aforementioned multi display method may be embodied as a programincluding an algorithm executable in a computer, and the program may bestored in and provided by means of a non-transitory computer readablemedium.

The computer-readable medium can include a computer-readable recordingmedium and a computer-readable transmission medium. Thecomputer-readable recording medium is any data storage device that canstore data as a program which can be thereafter read by a computersystem. Examples of the computer-readable recording medium include asemiconductor memory, a read-only memory (ROM), a random-access memory(RAM), a USB memory, a memory card, a Blu-Ray disc, CD-ROMs, magnetictapes, floppy disks, and optical data storage devices. Thecomputer-readable recording medium can also be distributed over networkcoupled computer systems so that the computer-readable code is storedand executed in a distributed fashion. The computer-readabletransmission medium can transmit carrier waves or signals (e.g., wiredor wireless data transmission through the Internet). Also, functionalprograms, codes, and code segments to accomplish the present generalinventive concept can be easily construed by programmers skilled in theart to which the present general inventive concept pertains.

According to various exemplary embodiments of the present generalinventive concept, it is possible to perform unstructured independentinputting through each display of a multi display apparatus 100, andcombine each input to perform an output corresponding thereto.

Although exemplary embodiments of the present general inventive concepthave been described above, it will be understood that the presentgeneral inventive concept is not limited to these exemplary embodiments.For example, the multi display 190 of the multi display apparatus 100does not necessarily include displays disposed on separate bodies whichare physically connected to one another. For example, the first display190 a and the second display 190 b may be disposed on opposite sides ofa single body, similarly to the configuration illustrated in FIG. 36,but without the hinge 185, and with both displays active. Thisconfiguration would allow the multi display apparatus 100 to be mademore compact. Furthermore, since user manipulations may be made in anondecisive location on the first display 190 a to affect operations ofthe second display 190 b, the user does not need to see the firstdisplay 190 a in order to control the second display 190 b. For example,the first display 190 a could be a touch pad that does not display ascreen, and instead receives user manipulations which may modify theoperation of user manipulations on the second display 190 b, asdescribed above. This configuration would allow a user to efficientlyperform operations on the second display 190 b without needing to hold aseparate first display 190 a.

Furthermore, the displays of the multi display 190 do not need to bephysically connected, and may communicate wirelessly with each otherthrough communication module 110, thereby allowing a greater degree offlexibility in the positioning of the displays. The controller 130 maydetermine a relative location of each display with the GPS module 115 orgeomagnetic sensor 152 for example. This would allow a user of onedisplay to perform user manipulations relative to another display. Forexample, a user may perform a drag operation on first display 190 b inthe direction of second display 190 b, to perform an operation on seconddisplay 190 b.

In the case of a plurality of displays, manipulations on one display canaffect multiple other displays. For example, as illustrated in FIG. 48,a user of first display 190 a in wireless communication (represented bythe double arrow) with second display 190 b and a third display 190 cmay perform a manipulation (a touch and hold operation) to capturecontent displayed on the first display 190 a (see FIG. 48 view 1).Subsequently, user manipulations (for example, a double tap with aninput pen 200) performed on second display 190 b and third display 190 cmay insert the content into each corresponding display as a memo 2110(see FIG. 48 view 2).

Although a few embodiments of the present general inventive concept havebeen shown and described, it will be appreciated by those skilled in theart that changes may be made in these exemplary embodiments withoutdeparting from the principles and spirit of the invention, the scope ofwhich is defined in the appended claims and their equivalents.

What is claimed is:
 1. A multi display apparatus comprising: a firstbody comprising a first display; a second body comprising a seconddisplay; a hinge configured to rotatably connect the first body and thesecond body to support the first body and the second body; a sensorconfigured to sense a first user manipulation made in a nondecisivelocation of a nonstructured first screen displayed on the first displayand to sense a second user manipulation made in a nondecisive locationof a nonstructured second screen displayed on the second display; and acontroller configured to combine the sensed first user manipulation andthe sensed second user manipulation to perform an operationcorresponding to a result of the combination.
 2. The multi displayapparatus of claim 1, wherein: the controller changes a functionmatching the second user manipulation according to the first usermanipulation; and the controller performs an operation corresponding tothe second user manipulation according to the changed function, when thesecond user manipulation is sensed.
 3. The multi display apparatus ofclaim 2, wherein: the second user manipulation is a line input by a dragoperation, and; the controller increases a thickness or transparency ofthe line input of the second user manipulation when the first usermanipulation is a pinch out input; the controller decreases a thicknessor transparency of the line input of the second user manipulation whenthe first user manipulation is a pinch in input; and the controllerchanges a texture or thickness of the line input of the second usermanipulation when the first user manipulation is a touch and rotateinput.
 4. The multi display apparatus of claim 1, wherein: the firstuser manipulation is a touch and hold input and the second usermanipulation is a touch input; and the controller displays a menucorresponding to the second user manipulation when the first usermanipulation and the second user manipulation are sensed at the sametime.
 5. The multi display apparatus of claim 1, wherein: the controllerchanges the second screen displayed on the second display according tothe first user manipulation when the first user manipulation is sensed;and the controller displays the second screen changed by the first usermanipulation according to the second user manipulation, when the seconduser manipulation is sensed.
 6. The multi display apparatus of claim 5,wherein: the controller expands the second screen displayed on thesecond display when the first user manipulation is a pinch out input;the controller reduces the second screen displayed on the second displaywhen the first user manipulation is a pinch in input; and the controllerrotates the second screen displayed on the second display when the firstuser manipulation is a touch and rotate input.
 7. A multi displayapparatus comprising: a first body comprising a first display; a secondbody comprising a second display; a hinge configured to rotatablyconnect the first body and the second body to support the first body andthe second body; a sensor configured to sense a first user manipulationmade in a nondecisive location of the first display and a second usermanipulation made in a nondecisive location of the second display, whenthe first display and the second display respectively display anonstructured first screen and a nonstructured second screen; and acontroller configured to combine the sensed first user manipulation andsecond user manipulation to recognize the combination as one usergesture, and to display an execution screen corresponding to the oneuser gesture on at least one of the first display and the seconddisplay.
 8. The multi display apparatus of claim 1, wherein the firstuser manipulation is an input proximate to or touching the first displayusing a user's body, and the second user manipulation is an inputproximate to or touching the second display using an input pen.
 9. Amulti display method of a multi display apparatus comprising a firstbody comprising a first display, a second body comprising a seconddisplay, and a hinge configured to rotatably connect the first body andthe second body to support the first body and the second body, the multidisplay method comprising: displaying a nonstructured first screen onthe first display; displaying a nonstructured second screen on thesecond display; sensing a first user manipulation made in a nondecisivelocation on the first screen and a second user manipulation made in anondecisive location on the second screen; and combining the sensedfirst user manipulation and the sensed second user manipulation andperforming an operation corresponding to a result of the combination.10. The multi display method of claim 9, wherein performing theoperation comprises: changing a function matching the second usermanipulation according to the first user manipulation; and performing anoperation corresponding to the second user manipulation according to thechanged function when the second user manipulation is sensed.
 11. Themulti display method of claim 10, wherein: the second user manipulationis a line input by a drag operation; and changing the functioncomprises: increasing a thickness of the line input of the second usermanipulation when the first user manipulation is a pinch out input;reducing a thickness of the line input of the second user manipulationwhen the first user manipulation is a pinch in input; and changing atexture of the line input of the second user manipulation when the firstuser manipulation is a touch and rotate input.
 12. The multi displaymethod of claim 9, wherein: the first user manipulation is a touch andhold input and the second user manipulation is a touch input; andperforming the operation comprises: displaying a menu corresponding tothe second user manipulation when the first user manipulation and thesecond user manipulation are sensed at the same time.
 13. The multidisplay method of claim 9, further comprising: changing the secondscreen displayed on the second display according to the first usermanipulation when the first user manipulation is sensed, whereinperforming the operation comprises: displaying the second screen changedby the first user manipulation according to the second usermanipulation, when the second user manipulation is sensed.
 14. The multidisplay method of claim 13, wherein changing the second screencomprises: expanding the second screen displayed on the second displaywhen the first user manipulation is a pinch out input; reducing thesecond screen displayed on the second display when the first usermanipulation is a pinch in input; and rotating the second screendisplayed on the second display when the first user manipulation is atouch and rotate input.
 15. A multi display method of a multi displayapparatus comprising a first body comprising a first display, a secondbody comprising a second display, and a hinge configured to rotatablyconnect the first body and the second body to support the first body andthe second body, the multi display method comprising: sensing a firstuser manipulation made in a nondecisive location of a nonstructuredfirst screen displayed on the first display; sensing a second usermanipulation made in a nondecisive location of a nonstructured secondscreen displayed on the second display; combining the sensed first usermanipulation and second user manipulation to recognize the combinationas one user gesture; and displaying an execution screen corresponding tothe one user gesture on at least one of the first display and the seconddisplay.
 16. The multi display method of claim 15, wherein the firstuser manipulation is an input proximate to or touching the first displayusing a user's body, and the second user manipulation is an inputproximate to or touching the second display using an input pen.